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The Management Assessment Center (MAC) o£ the Dade 
County (Florida) Public Schools is a unique project/ employing 
multiple techniques to evaluate behavior for school-level 
administrator personnel selection. This final report of an evaluation 
deals primarily with issues of the validity and the utility of the 
MAC* To ascertain validity of the WkC,^ the performance of candidates 
was correlated with their subsequent job performance. The data 
analysis of the results revealed that the validity correlations were 
positive and statistically significant. The evaluation notedi (1) 
inter-rater reliability was high; (2) the validity correlations were 
substantially higher than those generally produced by the interview 
method; (3) the validity correlations compared favorably with 
assessment centers; and (4) there is evidence that the validity 
correlations are still rising. Thus, it was concluded that the MAC 
does predict job performance. To investigate whether the resources 
allocated to the MAC are a worthwhile investment in the improvement 
of the selection process^ the evaluation compared the results of the 
present to the former process* The results revealed that the 
interview MAC selection process is not superior to the interview-^only 
process* The evaluation recommended that the minimum passing score of 
the MAC be raised to improve the effectiveness of the existing 
process. An appendis includes the M/R Rating Instrument. (BAE) 
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The evaluatlpn of the Mama^emen A^-^^B&r^mz Center (MAC) spanned three years 
and generated two reportis. The fi-j^t rri^port, which was titled Prallffllnary 
Report on the Eva luation c^ f l^y,_Ji^^M^^ B^^ Asaeggment Center » was published in 
March of 1984, It dealt 7- UMgri ' y rth the practices and ^roeadures that con- 
atitute the operation of the m^Z. following document Is the second and 

final report. Wille the x.:^^-I port deals briefly with the operation of the 
MAC, it does not replica L t ' preMmlnary report. The final report deals 
priniarlly with the crltics^t 1 c i ^ ol the validity and the utility of the MAC- 
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EXECUTIVE SmmRY 



An astessmant center is an assessment method that employs multlpla techniques 
to evaluate behavior. The techniques can include written teats or interviews, 
but they are most often limited to Job simulation exercises. The subject's 
behavior is observed by a group of assessors, who pool their observations to 
form a final evaluation, While industry has utilized the assessment canter 
method for personnel selection since the 1950's, true assessment centers are 
relatively new In public education. For this reason, the Management Assess- 
ment Cemter OlAC) of the Dade County Public Schools is a unique project. 

The MAC was developed in 1982 by Assessment Designs, Inc*, a management con- 
sulting firm. The funds for the development of the MAC were provided by the 
state under the prdvisldns of the Management Training Act of 1981* The dis- 
trict, however, unde writes the annual operating budget of the MAC, which ex- 
cluding assessor time (approximately 520 days) is currently $94,982, 

The conceptual framework of the MAC Is based on a job analysis of the dis- 
trict's school-level administrators conducted by Assessment Designs. The job 
analysis identified the following nine skilli as necessary for successful Job 
performance: (a) leadership, (b) organising and planning, (c) perception, (d) 
decision making, (e) decisiveness, (f) Interpersdnal, (g) adaptability, (h) 
oral communication, and (1) written conmunicatlon* In order to assess these 
skills, three exercises were developed for the MAC, They include an in-basket 
exercise, a parent conference simulation and a teacher observation simulation. 

The primary function of the MAC is screening candidates for the Job of school- 
level administrator. Before a candidate can Interview for a vacant position 
of principal or assistant principal, he/she must demonstrate through the MAC 
exercises the ability to sucoessfully perform the Job, Successful performance 
at the MAG means obtaining a minimum score of four on a seven-point rating 
scale for each of the nine skills. The skill ratings are provided by incum- 
bent administrators (pay grade 43 or higher), who are specially trained to 
function as MAC assessors. The skill ratings are the composite Judgement of 
three assessors, who observe the candidate's performance on the exercises* 
(For more detailed information on the MAC procedures, see the subsection ti- 
tled Management Assessment Center on pages 5 - 6.) 

The principal focus of the evaluation of the IWZ was the validation of the 
process. Validation basically involves accumulating aufricient data on the 
process and its outcome to warrant confidence in decisldns based on it. The 
validation of the MAC process was mandated by both legal and fiscal considera- 
tions. In reference to the legal consideration, personnel selection methods 
have repeatedly been challenged In the federal courts on the grounds of "ad- 
verse impact". Adverse impact is a situation where a personnel selection 
method worka to the disadvantage of a legally protected race, sex or ethnic 
group. While assessment centers have been legally challenged less often than 
some other personnel selection methods (e.g., paper and pencil tests), many 
assessment centers do exhibit adverse impact. The MAC is no exception. Al- 
though limited in degree, the MAC exhibits adverse impact in the categories of 
race and ethnicity, Md under the circumstances, legal prudence mandates that 
the validity of the B^C be documented. 
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In reference to the flacal consideration. It should be acknowledged that aa- 
sessment centers in general are more expensive than other personnel aelection 
inethods. In the interest of cost efficiency, the district must determine if 
the resources allocated to the MAC are a worthwhile Investment in the improve- 
ment of the selection of school-level administrators. The initial step in 
making this determination is the validation of the MAC. (For more detailed 
information on the validity issues concerning the MAC. see the section titled 
Validity of Assessment Centers on pages 9 - 11.) 

The evaluation of the MAC spanned three years and generated two reports, a 
preliminary report and this final report. The preliminary report, which was 
published in March of 1984, focused primarily on the MAC process. The report 
noted that during the first year of operation in 1981-82, the MAC had exper- 
ienced some start up problems. The MAC staff, however, had been vary respon- 
sive in addressing these problems, and thus had facilitated the subsequent 
development of the MAC. Consequently, the MAC assessors, who were in a unique 
position to observe the operation of the center, were very supportive of both 
the MAC staff and the MAC proeees. Indeed, the only significant problem in 
the MAC process Identified by the preliminary report was the center's passing 
rate which was found to be comparably high. (For more detailed Information on 
this phase of the evaluation, contact the Office of Educational Accountability 
and request a copy of Preliminary Report on the Evaluati on of the Manaeement 
Assessment Center . ) ~ ' — ' — — ^ 

Of greater Importance than the MAC process, however, is the intended outcome 
of the process, which is the prediction of a candidate's subsequent job per- 
formance. The degree to which the MAC achieves this objective is a measure of 
its validity as a personnel selection method. To ascertain the validity of 
the MAC, the performance of candidates at the MAC was correlated with their 
subsequent performance on the job. The data analysis of the results revealed 
that the validity correlations were positive and statistically significant. 
Moreover, the evaluation noted i (a) the inter-rater reliability, which Is 
considered a prerequisite to validity in an assessment center, was high| (b) 
the validity correlations were substantially higher than those generally' pro- 
duced by the interview method | (c) the validity correlations compared favor- 
ably, with those of other assessment centers | and, (d) there is evidence that 
the validity correlations are still rising. Thus, it was concluded that the 
MAC does predict job performance. (For more detailed Infcrmatlon on the con- 
clusions regarding the validity of the MAC, see the subsection titled Crite- 
rion-Related Validity of the MACi Conclusions on pages 44 - 46.) 

Beyond the question of validity is the question of the MAC's utility. In 
other words, are the resources allocated to the MAC a worthwhile investment in 
the improvement of the selection process for school-level administrators? In 
order _ to answer this question, the evaluation compared the results of the dis- 
trict's present selection process with the former selection process. The for- 
mer selection process essentially consisted of a series of interviews for the 
qualified candidates. The present selection process differs in the use of the 
MAC to screen the qualified candidates prior to the interviews. The results 
of the comparison revealed that, despite the validity of the MAC, the inter- 
view-MAC selection process is not superior to the interview-only selection 
process. Thus, under the existing operating procedures, the MAC has no util- 
ity. 
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This out coma ^ n#^erthelass, ia underscandable^ glvM thm minimum passing score 
of the M^G. The minimum paislng seofe of tha MAC is such that the few candi- 
dates who are eliminated ttom considaratlon ^would probably have bean elimin-^ 
ated anyway by the intervlwg . Under tha c^lrcumstancas, the interviews in 
affect bacome tha overriding factor In both s^^lactlon processas. Thus* it was 
concluded that there was no advantage in inco^^porating the MAC into the salee- 
tlon process ^ not becausa of a diflclaney in S,ts validity hut because its val-- 
idlty was essantlally not usedi (For more detailed information on the con- 
clusions regarding the utility of thi M^C, m^m the subsection titled Compari- 
son of the Selection Processes s Cennluaions csn pages 46 — 47.) 

Consequently s this avaluatlon ticonrainds that the minimum p:iasing score of tha 
MAC be raised. This upward adjtiitmant In the passing score should be done un« 
dar the direction of a qualified consultant * &±nce it will likely increase the 
adverse impact of the HACp Assuffllng an appro^priate adjustment in the passing 
score, the evaluation also recomnds that the& district retain the MAC as part 
of its selection process of school-level adoi^nlstrators. This reconmendation 
is based on the established validity of the b^^AC^ as well as the demonstrated 
competence of the mC staff* The MAC by emp-^aoylng a higher minimum passing 
score will improve the af £ ectlvenisi of the es&isting selection process* With-^ 
out such an adjustment* however ^ there is no advantage in retaining the MAC* 
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INTRODUCTION 



An assessment center Is a personnel selection mathod which has bee=3i used In 
Industry since the 1950' a. Current estimates of the number of aases^ment cen- 
ters m operation in the United States range from 300 to l.OOQ. The exact 
number Is difficult to determine due in part to confusion over tha definition 
of an assessment center. The term has often been Incorrectly used to refer to 
a physical facility or to any proiram of assessment. A true aa»«gsment cen- 
ter, however, conforms to the "Standards and Ethnical Considerations for 
Assessment Center Operations." This s.t of standards, which was en^dorsed by 
the Third International Congress on the Assessment Center Method in I975 and 
revised In 1980, defines an assessment center in the following oaniiee-- 1 

An assessment center consists of a standardized evaluation of behavior 
based on multiple Inputs. Multiple trained observers and teeijri_iques are 
used. Judgements about behavior are made, in part, from apecla]— ly devel- 
oped assessment simulations. 

These judgements are pooled by the assessors at an evaluation meeting 
during which all relevant assessment data are reported and dlscu_^sed, and 
the assessors agree on the evaluation of the dimensions and an^r overall 
evaluation that is made (Task Force on Assessment Center Standards, 1980, 
P • 35) . 

Thus, an assessment process Is not an assessment center. If: (a) it =relies on 
a single technique, (b) only a single assessor Is involved, or (c) ch^re is no 
pooling of data by the assessors. 

Assessment Centers in Education 

In the 1970's the assessment center method was Introduced into the field of 
public education. In 1975 the National Association of Secondary SchDol Prin- 
cipals (NASSP) In conjunction with the American Psychological Association de- 
veloped an assessment center model. The purpose of this pilot ptoiec^t was to 
demonstrate another approach to selecting potentially successful administra- 
tors that may set a standard for others to emulate" (Hersey, 1977). By 1984 
a total of 19 assessment centers had been patterned after the NASS3 modal' 
These assessment centers served more than 200 school districts In 15 states- 
and, according to the NASSP, several additional centers were In the planning 
stages (Ogawa, 1984). This project consequently represents the most extensive 
use of assessment centers In public education to date. 

In 1979, a three year validity study of the NASSP assessment Centers was un- 
dertaken by a team of Michigan State University psychologists heftdeS by Dr 
Neal Schmltt. The subjects of the study were 167 individuals vho Siad been 
assessed at seven separate assessment centers and subsequently tfcired as 
school-level administrators. When the group had achieved a job tenure of up- 
ward to four years, job performance ratings of each subject were obtalDied from 
the immediate supervisor, two teachers, two support staff employaas. and the 
subject. The criterion Instrument utilized by these raters was a per=eormanee 
scale that assessed 15 job dimensions of a school-level administrator The 
scale was developed for the study from a national job analysis of principals 
and assistant principals conducted by Schmltt and his associates Tfae results 
of the study included a number of significant, positive validity CorreZ^Latlons, 
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and Schmitt concluded i 



ExEmination of the criterion-related validity of the assessment center 
indicated that there were positive relationships between assessment cen-- 
ter skill ratings and ratings of subsequent job performance (Schtnittj 
Noe, Meritt, Fitzgerald & Jorgensen^ 1983* pg. 2), 



Management Assessment Center 

The Management Assessment Center (MAC) of the Dade County Public Schools is an 
assessment center that has been used since 1982 by the diB':rict to select prin^ 
cipals and assistant principals. Although there are similarities between the 
MAC and the NASSP assessment center model* the MAC was d&^eloped indapendantly 
for the district by Assessment Designs, Incs , a private consulting firm. The 
funds for the development of the MAC were provided by the state under the pro- 
visions of the Management Training Act of 1981. The state also provided the 
funds for the subsequent development of two additional sets of parallel exer^ 
cises. The district, however, underwrites the annual operating budget of the 
MAC* which according to the Supervisor of the HAC is currently $94,982. This 
sum Includes the cost of materials and supplies ($1,037), the cost oL substitute 
services for MAG candidates currently holding a teaching position ,($8,150), and 
the salaries of the MAC staff including fringe benefits ($85,795) The staff 
consists of a secretary (pay grade 19), and a supervisor (page grade 44) who re- 
ports to the Executive Director of the Bureau of Staff Development* 

The conceptual framework of the MAC is based on a job analysis of Dade County 
school-level administrators conducted by Assessment Designs. The job analysis 
determined that the following nine skills are necessary for successful job per- 
formance i (a) leadership, (b) organizing and planning, (c) perception, (d) de- 
cision making, (e) decisiveness, (f) interpersonal, (g) adaptabiilty , (h) oral 
communication, and (1) written communication. In order to assess these skills, 
three exercises were developed for the MAC* They include an in-basket exercise, 
a parent conference simulation, and a teacher observation simulation. During 
the first day of the two-day assessment process, each candidate's performance on 
these exercises is observed by three Incumbent administrators (pay grade 43 or 
higher) who are specially trained to function as MAG assessors. On the second 
day, the three assessors pool their observations and through consensus rate the 
candidate -s performance. In order to be successful, a candidate must obtain an 
average rating of four or higher on a seven point scale for each of the nine 
skills. Following the rating of the candidates the assessors prepare the final 
report, and subsequently one of them meets with the candidate to review the re- 
port* 

The primary function of the MAC is the screening of the candidates for the jobs 
of principal and assistant principal. Essentially, the MAG is one step in the 
selection process that includes the following five steps i (a) the candidate ob- 
tains administration/supervision certification from the state, (b) the candidate 



Not included in the sum is assessor time ($104*000* based on 520 days at an 
average daily rate of $200), since it represents no additional cost to the dis- 
trict* 



takes the Technical Skills Test for School-Level Administrators^ (e) the can- 
didate is assessed at the MAC* (d) the candidate who is successful at the MAC is 
placed in a pool of eligible candidates , and (e) the eligible candidate la free 
to interview for appropriate open positions for a period of two years* Thus, 
before a candidate becomes eligible to interview for a job, the candidate must 
demonstrate in the Job simulations of the the ability to successfully per- 

form the job. 



Assessment Cycles of the MAC 

An assessment cycle refers to the operational period of the MAC during a 
school year. The first assessment cycle occurred during 1981-82* There has 
subsequently been an assessment cycle every school year for a total of four 
cycles. Table 1 displays the number of candidates assessed per cycles as well 
as the success rate. A review of the table reveals that the success rate for 
all candidates peaked during cycle 2 at 72.71, and it has dropped to its low- 
est point of 64.01 during the most recent assessment cycle (Bureau of Staff 
Development, 1985). This trend Is encouragingi because, as noted in the pre- 
liminary report, the success rate at the MAC appears to be much higher than the 
success rate of assessment centers described in the literatura (Office of Educa- 
tional Accountability, 1984). 



Table 1 

Success Rate pa^ Assessment Cycle 



Candidates 





Job 


Aseeesed 


Successful 


Success Rate 


Cycle 1 
(1981-82) 


Assistant Principal 
Principal 


205 
120 


132 
91 


64.3% 
75.8% 




Total 


325 


223 


68,6% 


Cycle 2 
(1982-83) 


Assistant Principal 
Principal 


240 
90 


170 
70 


70.8% 
77.7% 




Total 


330 


240 


72.7% 


Cycle 3 
(1983-84) 


Assistant Principal 
Principal 


179 

68 


123 
48 


68.7% 
70.5% 




Total 


247 


171 


69.2% 


Cycle 4 
(1984-85) 


Assistant Principal 
Principal 


208 

70 


128 
50 


61.5% 
71.4% 




Total 


278 


178 


64.0% 



This district test is currently used as a diagnostic tool rather than an em- 
ployment test. Candidates are simply informed of the results and advised on 
remediation. The results are not used to screen candidates. 
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Another method of axaraining the success rate of the MAC Is to focus not on a 
point in time, i*e,* an assessment cycles but rather on a group of candidates* 
Since the candidates have more than one opportunity to be assessed at the MAC, 
it is more realistic to compute the success rate based on the eventual success 
of a designated group of candidates. The selection of a designated group of 
candidates for this purpose is circumscribed by two factors: (a) the provisions 
of Board Rule 6Gk13-4A-1 . 13 , Assignment, Transfer and Appointment - Principals 
and Assistant Principals ; and (b) the remediation practices reconmiended for the 



Table 2 

Success Rata per Candidate Group 



Candidates 











Jnb 


Assessed 


Successful 


Suocess Rate 


Cycle 


1 




2 


Assistant Principal 


205 


171 


83*4% 










Principal 


120 


113 


94a% 










Total 


325 


284 


87.3% 


Cycle 


2 


& 


3 


Assistant Principal 


240 


176 


73.3% 










Principal 


90 


74 


82.2% 










Total 


330 


250 


75.7% 


Cycle 


3 


& 


4 


Assistant Principal 


179 


133 


74.3% 










Principal 


68 


54 


79.4% 










Total 


247 


187 


75.7% 



unsuccessful candidates. In reference to the first factor. Board Rule 
6GX13-4A-1. 13 states that the MAC candidates "assessed unacceptable twice... 
shall not be eligible to reapply for a period of two years following the last 
assessment." This means that a candidate who was unsuccessful in cycles 1 and 2 
cannot be reassessed until cycle 5. Since cycle 5 is scheduled for 1985-8& at 
the earliest, is not necessary to accommodate a candidate in this situation In 
identifying a designated group of candidates. In reference to the second fac- 
tor, the remediation practices reconmended for the unsuccessful candidates en-- 
courages them to be reassessed In the cycle imiedlately following the initial 
assessment. While It is possible for an unsuccessful candidate to be reassessed 
in some other subsequent cycles the Supervisor of the ^C has noted that this 
rarely occurs. Therefore, In keeping with these two factors, the designated 
group of candidates is defined as those candidates being assessed for the first 
time In any given cycle| and the method of computing the group's success rate is 
based on the results of this assessment cycle plus the results of the reassess- 
ment during the following cycle. 

A comparison of Table 2 with Table 1 reveals that the success rates computed by 
this method are higher than the success rates based on the assessment cycle. 
For example, the success rate for all candidates based on the group ranges from 
75.7% to 87.3% while the rate based on the assessment cycle ranges from 64.0% 
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to 72, 7% (Bureau of Staff Development, 1985). Both methods, however, reveal the 
same trend of decreasing succeas rate. Nevertheless, the success rate* computed 
by either method remain comparably highs since "in a typical assessment center 
only about half of those assessed are rated acceptable" (Cascio, 1982). 
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VALIDITY OF ASSESStffiNT CENTERS 



The basic focus of the present evaluation is the validity of the Management 
Assessment Center (MAC) . While the process of validating a personnel selection 
method like the MAC is technically compleKj the concept of validity is intui- 
tively simple. It begins with the premise that most decisions involving person- 
nel selection are made with something less than complete Information. Since a 
degree of uncertainty Is always present, enough must be known about the selec- 
tion method to reduce the uncertainty to an acceptable level. Thus^ a selection 
method is termed "valid" when sufficient knowledge has been accumulated about 
its process and outcome to warrant confidence in a decision based on it. 

There are two fundamental reasons for establishing the validity of the ^^C. The 
first reason is the legal issue, "toployee selection procedures have been chal- 
lenged in Federal Courts literally hundreds of times and have not been upheld 
about half the time they have been challenged" (Boehm, 1982). Most of these 
challenges have been based on Title VII of the Civil Rights Act* the Office of 
Federal Contract Compliance ^ecutive Order 11/46* and the 14th Amendment of the 
U,S, Constitution* These regulations and the numerous court rulings on the 
matter have led to the establishment of the Uniform Guidelines on Egiployee 
Selection Procedures by the Equal Employment Opportunity~CoBmission~(EEOC) . The 
current Issue of the Unlfom Guidelines was publithed August 25* 1978* and it 
has been endorsed by the major regulatory agencies* 

According to the Uniform Guidelines * it is advisable to document the validity of 
an assessment center when the results of the center exhibit adverse impact. Ad- 
verse impact occurs when there is a personnel selection rate that "works to the 
disadvantage" of any legally protected race* sex or ethnic group* If a suit is 
brought against an organization's personnel selection practices* the enforcement 
agencies will generally resort to the Uniform Guidelines "80% rule of thumb" to 
detect adverse impact. The rule of thumb is defined asi 

A selection rate for any race, sex or ethnic group which is less than four 
fifths (or 80%) of the rate for the group with the highest rate will gener- 
ally be regarded .*. as evidence of adverse impact (p* 38297). 

It should be noted that this rule is not a legal definition of discrimination. 
It is rather a "practical device" that is used to detect adverse impact when a 
personnel selection method is legally challenged. 

The application of the 80% rule of thumb to the MAC reveals a degree of adverse 
impact* as Table 3 illustrates. Table 3 is a compilation of the total number of 
candidates assessed since c^cle 1, The candidates are divided by sex* ethnic- 
ity/race and the job sought. The application of the 80% rule of thumb reveals 



3 " 

It should be noted that courvi rulings sinc:e the publication of the Uniform 
Guidelines have altered the appiicabilitr of .^ams of the guidelines. ~ 

4_ 

Tbe Uniform Guidelines makes no provisions for checking adverse impact by 
various sex-ethnic/race groups* e,g** Black females. 
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adverse impact in four categories: (a) Black principal candidates, (b) Hispanic 
principal candidates, (c) Black ^ssistant principal candidates, and (d) Hispanic 
assistinnt principal candidates* It should be noted that in the second cate^ 
gory, i.e., Hispanic principal candidates, there would not have been adverse im-- 
pact had one more candidate been successful. However, a degree of adverse im- 



Table 3 



Success Rate by Sex and Ethnicity for Cycles 
pact of the MAC ~ ~ 



X Through Cycle 4 1 Adverse Im- 



Principal Candidates 



Kumber of Candidates 
Successful Candidates 
Success Rate 
80% Rule of Thumb 
Adverse Impact (X) 



Sex 



Ethnicity 



Male 


Female 


White 


Black 


Hispanic 


201 


147 


164 


120 


64 


140 


119 


139 


77 


43 


69.6% 


80.9% 


84*7% 


64. 1% 


67, 1% 


NPC 


HSR 


HSR, NPC 


75.6% 
X 


79.2% 
X 



Sex 



Ethnicity^ 



Assistant Principal Candidates 


Male 


Female 


White 


Black 


Hispanic 


Number of Candidates 




349 


483 


426 


275 


131 


Successful Candidates 




234 


319 


327 


155 


71 


Success Rate 




67.0% 


66.0% 


76.7% 


56.3% 


54.2% 


80% Rule of Thumb 




HSR, NPC 


98*5% 


HSR, NPC 


73.4% 


70.6% 


Adverse Impact (X) 










X 


X 


Note.HSH^ group with *' 
der Title VII of 


highest success rate 
the Civil Rights Act 


"l NPC^ 


''not a protected cla 


ss" un- 



pact is exhibited in the other three categories* 
document the validity of the MAC, 



Thus, it is legally prudent to 



The MAC* however, is only one step In the personnel selection process. There is 
a possibility that the adverse impact Is not carried through to the actual hir- 
ing of the candidates. This "bottom line'' check for adverse impact, which was 
tentatively endorsed by th© Uniform Guidelines , Is no longer sufficient. In 



For example, the calculation of the 80% rule of thumb for the Hispanic assis- 
tant principal candidates involves dividing the group's success rate of 54.2% 
by the highest success rate in this category, i.e., 76.7% for White assistant 
principal candidates. The answer is multiplied by 100 and the product is 
70.6%| since this is leas than SO.OX, it is evidence of adverse impact. 
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ConneGticut vs. Teal (1982), the U.S. Supreme Court ruled that an indlvldu.^A 
step within a personnel selection process could be challenged deapite a bottom 
line that exhibited no adverae Impact, Consequently, it is advlaable to vali- 
date any step in a personnel aelection process that exhibits adverse impact re- 
gardless of the bottom line outcome. 

The second fundamental reason for establishing the validity of the I^C is the 
fiscal issue. Any personnel selection method involves some eKpenditure by the 
organization J but assessment centers are by comparison more expensive. Many 
organizations employing assessment centers accept the expense* because studies 
have generally demonstrated the value of assessment centers in personnel 
selection. Nevertheless, faith in the method is not a sufficient reason for 
Justifying the expense of an assessment center. In the interest of cost effi- 
ciency, the district must detemine if the resources allocated to the MAC are 
a worthwhile Investment in the improvement of the selection of school=level 
adminlatrators. The initial step in making this determination la the valida-- 
tion of the MAC, 
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DESIGN OF THE EVALUATION 



The basic objective of the Managament Assessment Center (MAC) is the prediction 
of the candidates' subsequent job performance* The degree to which the MAC 
achieves this objective is a measure of its validity as a personnel seleetion 
tnethod (Division of Industrial-Organisation Psychology, 1980). The present 
evaluation ascertained the criterion-related validity of the by establishing 

a statistical relationship between performance at the J^C and subsequent per« 
formance_on the job. In addition, the evaluation compared the results of the 
district's present selection process for school-level administrators with the 
former selection process. The essential difference between the two is that the 
existing process incorporates the ^C. Therefore, the comparison sei-^ed to 
reveal the bottom line effect of incorporating the mc into the selection pro- 
cess. In order to accomodate both the crlterion-related validity and the com- 
parison of the selection processes, the MAC evaluation was partitioned into 
three phases^ (a) the interview and the survey of the MAC assessors* (b) the 
criterion-related validity of the MAC, and (e) the comparison of the selection 
processes. For the sake of clarity, each phase will be addressed separately in 
this section and in several subsequent sections of the report. 



Interview and Survey of the MAC Assessors i Design 

The Initial phase of the MAC evaluation involved the interview and survey of the 
WlC assessors. Both the interview and the survey were conducted after the com- 
pletion of assessment cycle 1 In 1981-82, The assessors were targeted for this 
inquiry, because they were in a unique position to provide insights into the 
operation of the MAC. Structured interviews were conducted with 12 select 
assessors (approximately 15% of the assessors at that time). The group included 
representatives from the central office, the area offices, and every school- 
level. The assessors were asked nine open-ended questions regarding the general 
operation of the MAC, Their responses served as a guide in the development of 
the survey instrument. All the MAC assessors (81 at that time) were forwarded 
the survey instrument, and 77.7% returned usable responses. 



Criterion-Related Validity of the ^Ci Design 

The second phase of the evalut^tion involved the criterion-related validity of 
the MAC, Criterion-related validity is a method of statistically verifying a 
probable relationship betwesn a predictor and a criterion. In this case, the 
predictor was the MAC skill ratings of a designated group of subjects, and the 
criterion was the subsequent Job performance r^i tings of these subjects. The re- 
lationship between these two variables was calculated by the Pearson product- 
moment correlation coefficient, l.e, , the Pearson Thus, the basic elements 
of the design of the second phase of the evaluation consisted of: (a) the pre- 
dictor, (b) the criterion, (c) the subjects and (d) the Pearson r. In the sub- 
sequent paragrphas, each element is individually examined* ^ 

The predictor, as previously noted, consisted of the subjects' skill ratings 
from the ^C, The skill ratings of each subject inulude an Individual score by 
exercise by skill for a total of 24 scores. This number is leas than the pro- 
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duct of the three exereiaes by nine skills , because every skill is not assessed 
in every exercise. As Table 4 illustrates , adaptability is not assessed In the 
in^basket exercise ; leadership is not assessed in the parent conference exer- 
cise • and written cosmiunication is not assessed in the teacher obserg^ation exer- 
cise* In addition to these individual 24 scoresi each subject receives nine 
overall skill scores consisting of the weighced average of the individual scores 
within a. skill. Each of the individual and overall skill scores is based on the 
seven-point rating scale illustrated in Table 4. Successful performance re- 
quires that the subjects obtain a minimum score of four on each of thfi nine 
overall skill scores* 

The criterion, as previously noted, consisted of the subjects* Job performance 
ratings. The Job performance ratings were obtained by means of two instruments, 
the Job Performance SMle and the Effectiveness Scale (see Appendls) . The Job 
Performance Scale is the instrument developed by the team of Michigan State 
University psychologists headed by Dr. Neal Schmitt for the validity study of 
the NASSP assessment centers (Schmitt et al, , 1983)* This instrument is a be- 
havlorally anchored, slK-polnt rating scale which assesses the 15 job dimensions 
of a school-level administrator listed in Table 5. The instrument was deemed 
appropriate for this evaluation because it was developed from a nr^tional Job 
analysis of school level administrators. The use of this Instrument, further- 
more, afforded an opportunity £o compare the results of the MAC evaluation and 
Schmitt et al. (1983) study. The second instrument employed was the Effective- 
ness Scale. This instrument was developed specifically for this evaluation by 
Dr. Larrg^ Skurnlks an industrial psychologist formerly employed by the 
dlstrictp The instrument was baaed on Assessment Design^s job analysis as re- 
flected in both the ejcercises and the skills assessed at the MAC, A five-polnn 
rating scale is used in the instrument to assess the 14 Job dimensions of a 
school-level administrator listed In Table 5, In addition, the Instrument in- 
corporates an overall rating item designed to evoke a holistic response. This 
item differs from the other items in the instrument in the use of a stanine 
scale. Consequently, this single item was treated in the evaluation as a separ- 
ate instrument under the operational name of Overall Rating Item. 

The MAC evaluation spanned the two school years of 1982-83 and 1983-84. The job 
performance ratings of the subjects were obtalm^.d in June of each school year. 
The ratings for each subject were provided by four individuals who were in a 
position to obs.irve the subject *s Job performance. They included i (a) the sub- 
ject's supervisor (an area director if the subject was a prlncipali a principal 
if the subject was an assistant principal), (b) a support person (an assistant 
principal randomly selected)^ (c) two subordinates (two teachers randomly se- 
lected) ^ and (d) the subject him/herself. To Insure objectivity, a coding sys- 
tem was used to keep the ratings confidential. Moreover^ each rater had the 
option of not responding to any item in the instruments that requested Informa- 
tion he/she could not provide. These factors probably contributed to the high 
response rate. The composite number of raters returning usable criterion rat- 
ings was 91.81 in 1983 and 92.2% in 1984. 



^Dr. Skurnik is presently employed by Educational Testing Service, 



Table 4 

Skill^EKerciae Rating HatriK of the MAC 



Skill/Exarcise 



In-Sasket 



1, Leadership 

2^ Orgaiiizing and Planning 

3* Perception 

4* Deeision Making 

5« Deeisiveness 

6, Interpersonal 

7* Adaptability 

8* Oral Coiomunieation 

9^ Written Conmtinication 



2 3 

Parent Teacher Overall 

Conference Observation Skill Score 



MAC Rating Scale 
7 = Outstanding 
6 - Well above satisfactory 
B ^ toove satisfactoiY 
4 ^ iatisfactory 
3 ^ Below satisfactery 
2 m P?ell below satisfactoiy 
1 - Weak 



This skill is not assessed in this exercise. 
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Table 5 



Job Dimensions Assagsed by the Criterion Instruments 



Job Parformance Scale 



1. 


Curriculum and Instructional Leadership* 


Monitoring Curriculum Objectives 


2 # 


Curriculum and Instructional Leadership i 


Monitoring Individual Progress 


3. 


Coordination of Student Activities i Superviaion 


4. 


Studenf Activities s Participation 




5. 


Direction of Support Sarvices 




o * 


Support Services 1 Directing the Behavior: 


of Students 


?• 


Staff E^mluation 




8, 


Developmental Activities 




9. 


Community Relations 




10, 


Interpersonal Effectiveness 




11. 


Community Relations i Parents 






Coordination with District and Other Schools 




Fiscal or Monetary Management 






Maintenance of School Plant 






Structures Communication Wiich Provide for 


Cooperation Among Various Groups 


-™ . 


in School 






Effectiveness Sc^ 


tie 


1. 


Adaptability 


Oral CoMunication 


2, 


Conferring with Parents Q 


Paper Work 


3, 


Decision Making 10. 


Planning and Organizing 


4. 


Decisiveness 


Perception 


5, 


Interpersonal 12, 


Leadership 


6, 


Management Skill 13, 


Technical Itoow-How 


7. 


Observing Teachers 14, 


Written CoBmunication 
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The subjects in this phase of the evaluation consisted of 121 incumbent school^ 
leyAl administrators, who had been candidates at the MAC during cycles 1, 2 and 
3-* This group included 101 incumbents who had been auecessful at the MAC and 
subsequently hired by March of 1984| and 20 incumbents who, although unsuccessful 
at the mc, were incumbent school-level administrators. These 20 incumbents con- 
sisted of assistant principals who had been assessed for the Job of principal. 
Their inclusion in the group of subjects provided representation of the unsuc- 
cessful candidates, who constitute approximately 30% of the assessed popularion 
(see Table 1). In addition, it lessened the restriction in the range of the mc 
ratings which would have resulted in a suppression of the correlations between 
the predictor and the criterion. 

The Pearson was used to compute the correlations between the predictor and the 
criterion, i.e., the validity correlations. The Pearson £ is a formula for cal- 
culating the degree of linear relationship between two variables. The formula 
yields a correlation coefficient which ranges from - 1.00 to +1.00- A positive 
correlation coefficient indicates that the high values of the predictor (the 
first variable) are associated with the high values of the criterion (the second 
variable) and /or the low values of the predictor are associated with the low 
values of the criterion, A negative correlation coefficient indicates an Inverse 
relationship; in other words, the high values of the predictor are associated 
with low values of the criterion and/or vice versa. A correlation coefficient 
near zero indicates that there is no relationship between the predictor and the 
criterion. 

The Pearson t_ was used to compute two sets of validity correlations. The first 
set involved the entire group of 121 subjects, and it was computed using the job 
performance ratings obtained in June of 1984, Since this set of validity corre- 
lations is based on the last performance ratings of all the subjects. It repre-» 
sents the most crucial statistical relationship in this phase of the evaluation. 
The second set of validity correlations Involved the 47 subjects hired during 
1982-83, and it was computed using first the job performance ratings obtained in 
June of 1983, then those obtained in June of 1984, A comparison between the 1983 
and 1984 validity correlations was done to Identify the longitudinal trend in the 
values of the coefficients. The longitudinal trend is of interest, because sev- 
eral studies have noted a tendency of the values of the validity correlation co- 
efficients to rise over time (Bray & Grant, 1966| Mltchel, 1975| Hinlchs, 1978). 
In addition to the validity correlations, the data analysis of this phase of the 
evaluation Included the following procedures i (a) the intercorrelatlons of both 
the predictor and the criterion, (b) the inter-rater reliability of the predic- 
tor, (c) the correlations of consistency of the criterion, and (d) a check for 
criterion contamination. 



Comparison of the Selection Processes i Design 

The third and final phase of the MAC evaluation involved a comparison of the dis- 
trict's present selection process for school- level administrators with the former 
selection process. The fomer selection process essentially consisted of a ser- 
ies of interviews for the candidates. The present selection process differs in 
the use of the MAC to screen the candidates prior to the interviews. The focus 
of the comparison was the subsequent job performance of the candidates selected 
by these two respective processes. 



The group originally consisted of 122 subjects, but the transfer of one incum- 
bent hired during 1982-83 to a non-schoQl-level administrative position reduced 
the number to 121. 
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The subjects In this phase of Che evaluation consisted of 109 Incumbent school^ 
level administrators. The interview-only process was used to select 62 of them. 
These 62 subjects were hired during 1981*82, the last year the Interview-only 
selection process was operational. The remaining 47 subjects were hired the fol- 
lowing school year using the MAC-interview selection process. Incidently, these 
47 subjects were the same subjects used in the longitudinal view of the validity 
correlations in the second phase of the evaluation.^ 

Since the Interview-only subjects had on an average one more year of job tenure 
than the t^C-interview subjects, the comparison was staggered in time to equate 
Che two groups on this factor. The June of 1983 job performance ratings of the 
interview-only subjects were compared to the June of 1984 ratings of the t^C-in- 
terview subjects. The method of rating the job performance of each group adhered 
to the same Instruments and procedures used in the second phase of the evalua= 
tion* 

It should be acknowledged that the design of this phase of the evaluation Is not 
a true experimental design , because the Initial distribution of subjects into the 
two groups was not achieved through randomigatioh. This was, of course, preclud- 
ed by the operational status of the MlC. Wille it would have been preferable to 
equate the two groups through randomization, no reason has been identified that 
would suggest that the two groups were characteristically different prior to 
selection. Not only Is it unlikely that the pool of potential candidates changed 
significantly from 1981-82 to 1982-83, but self-nomination was the initial step 
in both selection processes. 

Finally, the statistical comparison of the two groups was done by means of analy- 
sis of variance (ANOVA) , ANOVA is a method of dividing the variation of quanti- 
tative data into different parts in order to determine the source or cause* The 
relative magnitude of the variation from each source can then be assessed to as- 
certain whether It Is greater than expected. In this phase of the evaluation, 
^OVA was used to test the statistical slgnificanca of the difference between the 
means of the job performance ratings of the two groups. If the MAC-intervlew 
selection process is Indeed superior to the interview-only selection' process in 
the selection of better school-level administrators, the m^an of the job perform- 
ance ratings of the MAC-intervlew subjects should be significantly higher than 
the mean of the interview-only subjects. 
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RESULTS OF THE EVALUATION 



The results of the evaluation of tha Managemftnt Assessment Center (MAC) will be 
addressed in this section of the report. It should be noted that this section 
adheres to the same format employed in the Design of the Evaluation, in that the 
results of each of the three phases of the evaluation are addressed separately. 

Interview and Survey of the MAC Assessors: Results 

Since the interview and the survey of the ^lAQ assessors were conducted after the 
completion of assessment cycle 1 In 1981-82, the results of this phase of the 
evaluation were presented in the preliminary report which was published in March 
of 1984* These results , therefore, will not be replicated in this report. A 
reader who wishes to review the results of the first phase and/or to examine the 
Interview and survey Instrumeuts should cantact the Offliie of Educational 
Accountability and request a copy of Prallminary Report on the Evaluation of the 
Management^ Assessment Center . ~ ^ ^ ' ^ — ^ 

Criterion-Related Validity of the MACi Results 

The second phase of the evaluation involved the criterion-related validity of the 
MAC, The results of the computatloii of the validity correlations will be pre- 
sented in this section. In addition ^ the results of several data analysis pro- 
cedures encompassed by the design of this phaae will be included. Specifically, 
the results of the following procedures will be addressed i (a) the intercorre- 
lations o£ the predictor, (b) the Intercorrelatlons of the criterion, (c) the 
inter-rater reliability of the predictor, (d) the consistency of the criterion, 
(e) the validity correlations, (f) the longitudinal view of the validity corre- 
lations, and (g) the check fcr criterion contamination* 

Intercorrelatlons of the Predictor 

The intercorrelations of the predictor refers to the corralations between the 
skill ratings of the MAC. These Intercorrelations are important, because they 
indicate whether the skill ratings are providing unique information on the sub- 
jects' performance* Relatively low intercorrelations denote uniqueness, while 
high intercorrelations denote comiunallty. Conmiunallty implies that some of the 
skill ratings are tapping a common factor in the subjects' performance. Under 
such circumstances, it may be possible to delete some skills from the set without 
an appreciable loss of Information* 



The MAC evaluation revealed that the intercorrelations o 
moderate with the majority of the coefficients falling in 



£ the skill ratings are 
the 0.50's and 0*60's, 



8 - - ■ 

In describing the oorrelation coefficlBnts within this report, positive coeffi- 
cients in the. 0.40's, 0.50's and 0.60's are designated as "moderate"' coeffi- 
cients below or above this range are designated accordingly as "low" or "high". 
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Th^s modirate leval of intareorrelations , however i is not unexpected given the 
similarity of some of the skills* The overridlni consideration is whether the 
similarity of the skills resulted in such uniformly high intercorralation coeffi- 
cients aa to warrant the deletion of some skills from the set. A factor analysis 
of the nine skills revealed that this is not the c^ae. Although there is a de- 
gree of comunality, the nine skill ratings of the h^AC eisintially provide unique 
information on the subjects' performance. 

Int^arcorfalations of the Criterion 

Th« intereorrelations of the criterion refers to the correlations between the 
dli^ension ratings within each criterion instrumanc. Ai with the predictor, the 
prssence of uniformly high intareorrelations betwaam tha dimensions of a criter- 
ion Instrumant may danote communality, A more llkeLy intirpretation, however, is 
th«t it denotes tha aKlstance of a halo effect, A h^silo affect occurs whan a high 
rating on one dimension Influences tha rater to giv*« the subject comparable rat-- 
ine^ on the other dimensions. Since the criterion waters (unlike tha ^£AC assas- 
©O^s) ware not trained in observing and rating a subjict's performaaca, they 
coteld conceivably make such a mistake, Therefors, this Interpretatiom of uni- 
forTily high coefficients is logical. Yet it can be problematics for the subject 
utidlar scrutiny may indeed be superior in all dimensl ons, 

Thte MAC evaluation revealed that the dimension ratings provided by the subjects 
e3ch_iblt comparably low intercorrel^ion coef ficlan€» on both the Job Performance 
Sca^le and the Ef f r4Ctiveness Scale. This indicates that* compared to the other 
rat ^r groups t the subjects genarally saw greater dafferincas in their own par- 
for^ances on the various dimensions. This result ^ Ira^cidantly, If consistent with 
the findings of the Schmitt et al, (1983) study, St«condly, the dimension ratings 
pro— vlded by the other three rater groups exhibit «.mtercorrelation coefficients 
tha-^ ranga from moderate to moderate-to-high ^ but they are never so unifomly 
higai as to preclude the possibility of separats in^erpritations of the various 
ditt-ieniioni, Therefore, there Is no evidenca of uiia-ue influence exerted on the 
dimension ratings by a halo effect. This result is alio consistent with the 
findings of the Schmitt et al, <183) study, 

Int«r-Ratar Reliability of the Pradlctor 

The inter-rater reliability of the predictor is an l^idax of the degree of agree- 
ment iri thi skill ratings provided by the assessors, This index Is impor- 
tant s because in an assessment center high inter-rst^ raliability is con Idared 
a p Tarequislte to validity. The intar-ratar ralia&ility index of the MAC was 
coffl^utad using two methods, the Pearson ^ and the ooefjEicient alpha. Table 6 
dlg^lays tha Pearson correlations of the skill ^atinga provided by the three 
MAC assessors. Each assessor was arbitrarily dasigtiated Aiiessor 1, 2 or 3| and 



the* intercorrelations of the dimension ratings can n^ot be computed for the Over-- 
alL Rating Item, since this criterion instrumant con^^ists of a single dimension. 
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corrilatelons wera computad for each of the nine skills assessed at the MAC* An 
eKanlna^lon of Table 6 reveals that the correlationa are high with 62*91 of the 
coeffic^ants falling in the 0.80*s. These results are indicative of a high de= 
gree of agreement among the MAC assessors . 

Tatole 6 

Irifaar-Rater Reliability of the MAC Skill Ratings Based on the Pearson r 

1. Adaptability 
Aseessor 
1 

2 ,73 

3 .1^.1^ 
12 3 



1 

2 .90 

i .SS ,89 

12 3 



4* pecislon Haklng 
1 

2 ,88 

3 .82 ,86 

12 3 



2, &^z'ganiging and Planing 

A seessor 

1 

2 ,88 

3 .87 p87 

1 2 S 

A^ssessor 
1 

2 .87 

3 .81 .8i 

12 3 



5. Decisiveness 
Assessor 
1 

2 .85 

3 .SB ,87 

12 3 

€, Interpersonal 
Assessor 
1 

2 .77 

3 .71 .73 

12 3 



8^ Oral Communication 
Assessor 
1 

2 ,72 

3 , 74 , 76 

12 3 

i. Written CommuniGatlon 
Assessor 
1 

2 ,86 

3 .89 .86 

12 3 



Note. All Goefficients are significant at £ < 0,05 for N - lOi, 



20 



28 




Table 7 

Inter-Rater Raliabillty of the MAC Skill Ratings 3ased on Coefficient 
Alpha 



1* 


Leaderahlp 


.96 


2. 


OrganiEing and Planning 


.95 


3. 


Parception 


.94 


4. 


Decision Making 


.94 


5. 


Decisiveness 


.94 


6, 


Interpersonal 


.89 


7. 


Adaptability 


.89 


8. 


Oral ConmunlGation 


.89 


9* 


Written ComiunlQation 


,95 



Table 7 displays the coefficient alpha of the skill ratings provided by the MAC 
assessors. Coefficient alpha* which ranges In value from zero to ones is gener- 
ally utilised as a measure of agreement OTong items in a written test. In ap- 
plying coefficient alpha to this situatlf:-*, each MAC skill was regarded as a 
single test and each assessor as a single item within the test. An examination 
of Table 7 reveals that all the MAC skills exhibit a -coefficient alpha of 0.89 
or higher. As In Table 6* these results are indicative of a high degree of 
agreement among the MAC assessors. 

Since the Schmitt et al* (1983) study used the same two methods to compute the 
inter-rater reliability index, it facilitated a comparison of the results of the 
study and the MAC evaluation. Before the comparison Is made* however, two dif- 
ferences should be noted. First, since the NASSP centers routinely collect the 
data necessary to compute the inter-rater reliability index , Schmitt was able to 
draw from the centers' records and base his computation on 340 subjects. The 
MAC^ however* does not routinely collect this data* The inter-rater reliability 
index of the YikQ is based on only 106 subjects, because the data needed to com- 
pute the index was collected solely for this evaluation. Secondly, the NASSP 
assessment centers employ six assessors per assessmant, while the MAC amploys 
three. 

The comparison of the Schmitt study and the MAC evaluation reveal that the 
inter-rater reliability index computed by means of the Pearson r is higher for 
the MAC. Schmitt reported that "most of the correlations were in excess of 
0. 60'S while most of the MAC correlations were In excess of 0.80, In terms of 
the coefficient alpha, it should be acknowledged that the inter-rater reliabil- 
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Ity index computed by this mathod is enhanced by an Increase in the number of 
assessors (i.e., the items In the test). Nevertheless, despite the disparity In 
this regard J the results were quite similar. Schmltt reported a minimum coeffi- 
clent alpha of 0.90 for all skills, which is essentially equal to the 0.89 of 
the MAC. 

Consistency of the Criterion 

Table 8 displays the correlations between the composite ratings of the criterion 
Instruments (i*e.s the sum of all the dimension ratings within an instrument). 
These correlations provide information on the consistency of the ratings across 
the instruments. Thus, they reflect to a degree the reliability of the criter- 
ion. 

Table 8 

Correlations Between the Composite Ratings of the Criterion Instruments 
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(114) 


(115) 
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Note, All coefficients are significant at £ <^ 0*05 unless designated 
otheTOlsa. The n for each cell appears in parentheses, J ^ Job Perfor- 
mance Scale; E ^ Effectiveness Scalei 0 ^ Overall P*.atlng Item. 
^ot significant at £ <^ 0.05. 
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A review of Table 8 reveals that tha eonsistency correlations of the composite 
ratings provided by both the suparvisors and the support persons are generally 
high. By contrast, the correlations of the composite ratings provided by both 
the subjects and the subordinates exhibit some low correlations. These low cor-- 
relations, however, are not unexpected* In the case of the subjects, the low 
consistency correlations are generally a reflection of the previously noted low 
intercorrelations of the ratings provided by this group* In the case of the 
subordinates, the low consistency correlations are attributable to this group's 
propensity for opting not to respond to certain items in the instruments. Of 
the four rater groups, it should be noted that the subordinates invoked this 
option most often. This is understandable, given that the subordinates probably 
had the least direct contact with the subjects. While Invoking this option does 
not affect the correlations involving individual dimension ratings, a single 
"missing" dimension rating can adversely affect the correlations involving the 
composite ratings* For example, the composite ratings of the Job Performance 
Scale and the Overall Rating Item correlated only 0.09 (see Table 8, C), Yet, 
the correlations between the individual dimension ratings of these respective 
instruments are moderata with 80.0% of the coafflclents falling in the 0.40*s 
and 0*50's, 

In brief, the consistency correlations of the criterion Instruments displayed In 
Table 8 exhibit some low coefficients* These coefficients, however, are attrib- 
utable at least in part to a procedural factor in the computation* Generally 
the correlations in the table are high with 58,3% of the coefficiants falling in 
the 0*70's and 0*80's, This represents positive evidence of the consistency of 
the ratings across the criterion Instruments* 

Validity Correlations 

The correlations between the predictor and the criterion represent mast cru-- 

cial statistical relationship in this phase of the evaluation, '^hese correla- 
tions , known as validity correlations, are essentially a measure of how accu- 
rately the predicts the subjects- subsequent Job performance. In computing 
the validity correlations, the mC overall skill ratings of the subjects func- 
tion as the predictor, and the subjects' subsequent job performance ratings 
function as the criterion. Of the 121 subjects assessed, 47 were hired as 
school-level administrators during the 1982-83 school year, and 54 during the 
1983-84. In addition, the group included 20 subjects who, although unsuccessful 
at the MAC, were incumbent school-level administrators. The job performance 
ratings were obtained in June of 1983 for the original 47 subjects, and again in 
June of 1984 for the entire group of 121 subjects. 

The validity correlations based on the June of 1984 job performance ratings of 
all 121 subjects are displayed in Tables 9 through 12 and Tables 14 through 18. 
Specifically, Tables 9 through 12 display the correlations between the overall 
skill ratings of the MAC and the dimension ratings on the Job Performance Scale 
provided respectively by the supervisors, the support persons, the subordinates 
and the subjects. Tables 14 through 17 display equivalent data for the Effec- 
tiveness Scale; and Table 18 displays equivalent data for the Overall Rating 
Item, 

A review of Tables 9 through 12 reveals certain differences in the validity cor- 
relations based on the Job Performance Scale across rater groups. Focusing 
first on the supervisors, the validity correlations in Table 9 exhibit a posi- 
tive statistical relationship between the l^C overall skill ratings and the di- 
mension ratings provided by this group* Almost all the coefficieMs in this 
table are poeitlve, and 37*0% of them are statistically significant. 



A statistically significant relationship Is defined in this report as one that 
would occur only 5 times or less in 100 by pure chance, i,e,, £ <^ 0*05. 
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Likewisa the validity correlations in Table 10 exhibit a positive statistical 
relationship betwean the MAC overall skill ratings and the dimansion ratings 
provided by the support persons. Almost all the coef f iclentf, in this table are 
positive, and 32*5% of tham are stcitistically significant. 

Unlike Tables 9 and 10* tha validity correlations in Table 11 exhibit no statis- 
tical relationship betwean the MAC overall skill ratings and the dimension rat- 
ings provided by the subordlnatas on the Job Performance Scale. Although 62.2% 
of all the eoafficients in this table are positives very few of them are sta- 
tistically significant. Nevertheless, given the tendency of the values of val- 
idity correlations coefficients to rise over time* the preponderance of positive 
coefficients in Table 11 suggests that these validity correlations may in time 
exhibit a positive statistical relationship. 

The validity correlations in Table 12 exhibit no statistical relationship be- 
twaan the MAC overall skill ratings and the dimension ratings provided by the 
subjects on the Job Performance Bcala* The number of positive coefficients in 
this table is approKimately equal to the number of nagativa coefficients. 
Therefore, unlike Table 11, there is no evidence in Table 12 to suggest that 
these validity correlations may eventually axhiblt a positive statistical rala- 
tionship* 

Since the validity correlations of the Schmitt et al, (1983) study are based oh 
the Job Performance Scale, this affords an opportunity to compare tha results 
dapicted in Tables 9 through 12 with the results of the Schmitt study. Before 
this comparison is made, however, certain diffarances betwean the Schmitt study 
and the ¥AC evaluation should be noted. First, the rater groups consisting of 
support persons are not comparable. In the Schmitt study this group consisted 
of "secretaries, janitorial staff, ate*", while in the MAC evaluation it con- 
sists of assistant principals. Likewise, the rater groups consisting of subor- 
dinates are not comparable. In the Schmitt study this group consisted of desig- 
nated "senior level [teachers] who were likely to observe and be familiar with 
the [subject's] work'N Although in the MAC evaluation this group also consists 
of teachers, they were selected randomly. Finally, the job tenure of the sub- 
jacts hired after a favorable assessment is not comparable. In the Schmitt 
study the tenure of these subjects ranged up to four years, while in the yAC 
evaluation it only ranged up to two years. 

Despite this differance in tenure, an examination of Table 13 reveals that the 
proportion of positive and statistically significant validity correlations in 
the ^ylC evaluation compares favorably with the Schmitt study. Table 13 also re- 
veals that the MAC evaluation and the Schmitt study concur on the statistical 
relationship derived from the ratings provided by the two comparable rater 
groups, i,e,, the supervisors and the subjects. In the W^C evaluation, the val- 
idity correlations based on the supervisors' ratings exhibit a positive statis- 
tical relationship, while the validity correlations based on the subjects * rat- 
ings exhibit no statistical ralationship; the Schmitt study yielded the same 
results. 

By contrast, the MAC evaluation and the Schmitt study differ on the statistical 
relationship derived from the ratings provided by the rater groups which were 
not comparable, i,a,, the support persons and the subordinates. In the case of 
the support persons, the validity correlations based on this group's ratings ex- 
hibit a positive statistical relationship in the evaluation, but in the 
Schmitt study they eKhibit no statistical relationship. These different results 
are understandable, since the support persons in the MAC evaluation (i*e,, 
assistant principals) generally have a greater knowledge of the subject ♦s job 
and more direct contact with the subject than the support persons In the Schmitt 
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Table 13 



Comparison of the Validity Correlations Basad on the Job Performance 
Seale in the MAC Evaluation^ and the Sehmltt Study^ 



later Group/Study: 



Supervisors 
Support Persons 
Subordinates 
Stibjects^ 



mc 



ichaitt 



MAC Sclmitt 



% of Negative r 



0,7 

2.2 

37.? 
46.6 



19.4 
38.8 
21,1 
35.0 



% of Positive r 



99.2 
97,7 
62.2 
S3. 3 



10.5 
61,1 
7S.8 
65.0 



% of Negative r 
Significant at £ < 0,05 



% of Positiva r 
iignificant at £ < O.OS 



Supervisors 
Support Persons 
Subordinates 
Subjects^ 



0 
0 

1.4 
2,9 



Q,5 

2,2 
7.2 



37.0 
32.5 
2.9 
3,7 



46,6 
7.2 

20.0 
13.3 



The validity Gorrelations of the MAC evaluation consist of 135 coefficients per rater 
group, 

b 

The validity correlations of the Schmitt et al. (1983) study consist of 180 coefficients 
per rater group. 



Coraparable rater group. 
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study (l.e,, secretaries, janitors , etc.). In addition, the different results 
exhibited by the validity correlations based on the ratings provided the subor- 
dinates (1*6, s teachers) may also be attributable to a disparity in the degree of 
job knowledge and subject contact. The validity correlations based on the rat- 
ings provided by the. designated senior level teachers in the Schmitt study ex- 
hibit a positive statistical relationship • however, those based on the ratings 
provided by the randomly selected teachers in the MAC evaluation exhibit no sta- 
tistical relationship* 

Tables 14 through 17 are the next set of tables to be reviewed. These tables 
display the validity correlations based on the Effectiveness Scale, which is the 
second criterion instrument utilized in the MAG evaluation. Focusing first on 
the supervisors, the validity correlations in Table 14 exhibit a positive statis- 
tical relationship between the MAC overall skill ratings and the dimensions rat- 
ings provided by this group. Almost all the coefficients in this table are posi- 
tive, and 38,1% of them are statistically significant. Likewise, the validity 
correlations in Table 15 exhibit a positive statistical relationship between the 
MAC overall skill ratings and the dimension ratings provided by the support per- 
sons. Indeed, the validity correlations in this table eKhibit the strongest pos- 
itive statistical relationship in the MAC evaluation. All the coefficients in 
the table are positive, and 73,0% of them are statistically significant. 

Unlike Tables 14 and 15, the validity correlations In Table 16 eKhibit no statis- 
tical relationship between the MAC overall skill ratings and the dimension rat- 
ings provided by the subordinates on the Effectiveness Scale, Although 70,6% of 
the coefficients in this table are positive, very few are statistically signifi- 
cant. This preponderance of positive coefficients, which is similar to the re- 
sults on the Job Performance Scale (see Table 11), suggests that in time these 
validity correlations may exhibit a positive statistical relationship. 

The validity correlations in Table 17 exhibit no statistical relationship between 
the MAG overall skill ratings and the dimension ratings provided by the subjects 
on the Effectiveness Scale. The number of positive coefficients in this table is 
approximately equal to the number of negative coefficients. Therefore, unlike 
Table 16* there is no evidence in Table 17 to suggest that these validity corre- 
lations may eventually exhibit a positive statistical relationship. 

Table 18 is the last table to be reviewed in this section. It displays the val- 
idity correlations based on the Overall Rating Item, which is the third criterion 
instrument utilised In this study. Since this instrument consists of a single 
dimension, it is possible to display the validity correlations derived from all 
four rater groups in this single table, A review of Table 18 reveals that the 
validity correlations in the first and second columns, which are based on the 
dimension ratings provided by the supervisors and support persons respectively, 
exhibit positive statistical relationships. All the coefficients in these two 
columns are positive, and 66,6% of the coefficients In each column are statis- 
tically significant. By contrast, the validity correlations In the third and 
fourth columns, which are based on the dimension ratings provided by the subor- 
dinates and subjects respectively, exhibit no statistical relationships. 

In sumiary, the results of the MAC evaluation reveal that the validity correla- 
tions based on the supervisors* ratings exhibit a positive statistical relation- 
ship in all three criterion instruments (see Tables 9, 14 and 18). Furthermore, 
the validity correlations based on the support persons* ratings exhibit compar- 
able results (see Tables 10, 15 and 18), By contrast, the validity correlations 
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Table 14 



Correlations Between the MAC Overall Ski ll Racings and the Dimeniion Ratings Pro vidad by the Supervisors 
on the Ef fectlvaness Scale 



ikills/Diniensiona I 



i 



5 * 
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Q1 




f— 1 


1^ 






1* Leadtrahlp 


-.01 
(IPS) 


,15 
(106) 


.00 
(105) 


,08 
(105) 


.07 
{105) 


.07 
(104) 


.10 
(104) 


.16* 
(104) 


.08 
(104) 


-02 
(105) 


.07 
(105) 


.11 

(106) 


,01 
(105) 


.15 
(106) 


2* Organising and Planning 


.11 

[106) 


.19* 

ao6) 


.06 
(105) 


,11 
(105) 


.18* 
(105) 


.15 
(104) 


.22* 
(104) 


.29* 
(104) 


,19* 
(104) 


.09 
(105) 


.13 
(105) 


.20* 
(106) 


.06 
(105) 


.3S* 
(106) 


3, FGrception 


.12 
(106) 


,1B* 
(106) 


.09 
(105) 


.18* 
(105) 


,13 
(105) 


,15 
(104) 


.17* 
(104) 


.16* 
(104) 


.13 
(104) 


.12 
(105) 


.20* 
(105) 


.22* 
(106) 


.11 
(105) 


,25* 
(106) 


4* Deelslon Making 


.11 
(106) 


.20* 
(106) 


.12 
(105) 


.22* 
(105) 


.15 
(105) 


.IP* 
(104) 


.27* 
(104) 


.17* 
(104) 


.18* 
(104) 


.14 
(105) 


.17* 
(105) 


.22* 
(106) 


,13 
(105) 


.29* 
(106) 


B, Deolsivaness 


.01 
(106) 


.13 
(106) 


.00 
(105) 


.10 
(105) 


.09 
(105) 


.09 
(104) 


.15 
(104) 


,13 
(104) 


,09 
(104) 


.02 
(105) 


.08 
(105) 


.12 
(106) 


.03 
(105) 


.22* 
(106) 


6* Interpersonal 


,14 
(lOi) 


.13 
(106) 


ae* 

(105) 


.li* 

(101) 


.17* 
(105) 


,17* 
(104) 


.24* 
(104) 


.IP* 
(104) 


.14 
(104) 


.17* 
(105) 


.14 
(105) 


.14 

(106) 


.12 
(105) 


.24* 
(106) 


7, Adapt^illty 


.20* 
(106) 


.20* 
(lOS) 


.11 

(105) 


.11* 
(105) 


,2?^ 
(105) 


.22* 
(104) 


.25* 
(104) 


.24* 
(104) 


.15 
(104) 


.14 
(105) 


.20* 
(105) 


.24* 
(106) 


.14 
(105) 


.26* 
(106) 


8* Oral Communication 


.10 
(106) 


.13 
(106) 


,03 
(105) 


.11 
(105) 


,11 
(105) 


,16* 
(104) 


.13 
(104) 


.27* 
(104) 


.10 
(104) 


.09 
(105) 


,14 
(105) 


.15 
(106) 


,09 
(105) 


.23* 
(106) 


Written COTiunigation 


,04 

(lOi) 


.06 
(106) 


-.03 
(105) 


,04 
(101) 


-.01 
(105) 


,06 
(104) 


.12 
(104) 


,19* 
(104) 


,06 
(104) 


.04 
(lOS) 


.08 
(105) 


.09 
(106) 


.05 
(105) 


.26* 
(106) 



Note , The n for eaah ctll appears in parentheies, 
*£ < 0,05. 
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Table 15 

CorralatlDng Between the MC Overall Skill Rat ings and the Dim enaion Ratings Provided 
Peraong on the Effectiveness Scale 



by the Support 
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Skllls/Dlmensioni s 






























.32* 
( 73) 


.24* 
( 72) 


,25* 
C 73). 


.30* 
( 72) 


.24* 
( 73) 


.34* 
( 72) 


2* Oigdniilng and Planning 


.30* 
( 73) 


.24* 
( 72) 


.17 
( 73) 


.20* 
( 72) 


-26* 
( 71) 


-30* 
( 72) 


3, Perception 


.30* 
( 73) 


.26* 
( 72) 


-22* 
( 73) 


.21* 
( 72) 


.23* 
( 73) 


.24* 
( 72) 


4* Deeieion Making 


.31* 
( 73) 


.15 
( 72) 


.IS 

( 73) 


.19 
( 72) 


.27* 
( 73) 


.24* 
( 72) 


B, Decisiveness 


.29* 
( 73) 


.19 
( 72) 


.19 
C 73) 


.19 
( 72) 


.22* 
( 73) 


.25* 
C 72) 


6. Interpersanal 


.18 
( 73) 


.11 
{ 72) 


.13 
C 73) 


*13 
( 72) 


.14 
C 73) 


.22* 
( 72) 


?• Adaptability 


,34* 
( 73) 


.20* 
( 72) 


.26* 
C 73) 


.21* 
( 72) 


.30* 
( 73) 


,31* 
( 72) 


S, Oral Coramunlcatien 


.26* 
( 73) 


.27* 
C 72) 


ai 

( 73) 


.19 
( 72) 


,26* 
C 73) 


.27* 
( 72) 


9t Written CDmnunlcation 


.19* 
C 73) 


.21* 
( 72) 


.12 
( 73) 


.26* 
( 72) 


.15 
( 73) 


.26* 
( 72) 



N 

e 







o 
















dj 




u 




£- 










_i 








CM 




i-H 



.36* 
( 70) 

,33* 
( 70) 

.21* 
( 70) 

.25* 
C 70) 

.18 
C 70) 

.24* 
{ 70) 

.25* 
C 70) 

.26* 
C 70) 

.15 
( 70) 



.41* 
( 72) 

,37* 
( 72) 

,33* 
C 72) 

.27* 
C 72) 

.25* 
C 72) 

.15 
C 72) 

.26* 
( 72) 

-32* 
( 72) 

-2S* 
( 72) 



.34* 
( 73) 

.24* 
( 73) 

.22* 
C 73) 

.21* 
( 73) 

.24* 
( 71) 

.14 
( 73) 

.21* 
( 73) 

.24* 
C 73) 

.25* 
C 73) 



,32* 
{ 73) 

.26* 
( 73) 

.24* 
C 73) 

.24* 
C 73) 

.23* 
( 73) 

.23* 
( 73) 

.26* 
C 73) 

.33* 
( 73) 

.21* 
( 73) 



,27* 
( 72) 


.31* 
( 71) 


.31* 
( 72) 


.37* 
( 72) 


.16 
( 72) 


.25* 
( 71) 


.20* 
C 72) 


.36* 
( 72) 


,24* 
( 72) 


-22* 
( 71) 


,24* 
( 72) 


.31* 
C 72) 


.13 
( 72) 


,19* 
( 71) 


.16 
( 72) 


.26* 
( 72) 


,13 
( 72) 


.24* 
C 71) 


.15 
C 72) 


.28* 
C 72) 


.04 
( 72) 


.08 
( 71) 


.09 
( 72) 


.16 
( 72) 


.20* 
( 72) 


.23* 
( 71) 


.22* 
( 72) 


-27* 
( 72) 


,10 
{ 72) 


.22* 
( 71) 


,18 
( 72) 


.34* 
( 72) 


,04 
( 72) 


.19 
C 71) 


.17 
( 72) 


-31* 
( 72) 



The n for each cell appears in parentheses, 
*£ < 0,05, 
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Table 16 

Correlations Ee twean the Overall m]] % 
on Ehe Efigctlveness Scale 



atings and the DiniensiQn Ratings Provldidjb th. Subordinatt 



SWlls/DiBinsioni: 
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J- 



1, Leadifsklp 


.00 
(92) 


-.09 
(1?) 


.03 

(80) 


■,09 
(Ml 


.05 
(90) 


-.10 
(73) 


-.14 

(18) 


.08 
( 93) 


-.19 
(55) 


sOS 
( 52) 


-,04 
(73) 


.03 

( 71) 


.09 
(56) 


.00 
( 781 




.09 
(92) 


-.03 
(57) 


,12 
( 80) 


.03 
( 84) 


.07 
( §0) 


.03 
( 71) 


.00 

1 58) 


.21* 

( 93) 


-.08 
(551 


.08 
(52) 


.03 
( 73) 


,13 
( 78) 


.20 
( IS) 


,16 
( 78) 


3, Peremption 


.01 

(12) 


■,08 
(S?) 


.05 
(80) 


-.01 
(84) 


.02 
(90) 


{ 73) 


■.11 
{ 51) 


,03 
(93) 


-.17 
(iS) 


.01 

(52) 


-.03 
( 73) 


.05 
( 78) 


.12 

(56) 


■.01 
( 781 


4, Dicislon Hiking 


M 
( 92) 


.00 

(i?) 


.07 

(10) 


.03 
(14) 


.10 

( 90) 


-.01 
( 73) 


.00 
(18) 


.17* 
(93) 


".03 
(55) 


.04 
( 52) 


.05 
(73) 


.11 
( 78) 


.15 
(56) 


.10 
( 78) 


S* Dtcliivinesj 


.14 

(92) 


.00 

(1?) 


.15 
(SO) 


.04 
(84) 


.09 
(90) 


.01 
( 73) 


.00 
(38) 


.14 

(93) 


-.03 
(55) 


.07 
( 52) 


.05 
( 73) 


,09 
( 78) 


.10 

( 56) 


.12 

( 78) 


6, iBterpitionil 


.12 
(92) 


.00 
(57) 


.14 
(80) 


.04 

(§4) 


.04 

( 90) 


■,07 
{ 73) 


.03 
(58) 


.07 
(93) 


sl7 
(SI) 


.07 
(12) 


.12 
( 73) 


.10 
(781 


.11 
(IS) 


.01 
( 78) 


1. AdaptilsiUty 


.01 
(92) 


-.08 
(57) 


.06 
(80) 


-.10 
(§4) 


.04 
( 90) 


M9* 
(73) 


%10 
( 58) 


.11 
(93) 


-.20 
(55) 


-.05 
(52) 


.09 
( 73) 


-.05 
[ 78) 


-.05 
( 56) 


.03 
( 71) 


i, Qril Cooiunicitien 


,1S 
(92) 


.01 
( i7) 


.12* 
(80) 


.13 

( 14) 


.14 
( 90) 


.10 
( 73) 


.07 

( 51^ 


,17* 

(93) 


.09 

(55) 


.22 
(52) 


.19* 
( 73) 


.18 
( 78) 


.24* 

( 5i) 


.17 
( 78) 


1. Hritten Ccnunleation 


.10 

( 92) 


.04 
(57) 


.10 
(80) 


-.03 
(84) 


.07 

(90) 


,00 
(71) 


-.02 
( 58) 


.14 

(93) 


.01 

(55) 


.08 
(52) 


.03 
( 73) 


.OS 
(71) 


.10 

(ii) 


.12 
( 78) 



it|* Thi I for m cell ippiirs In pirintheses. 
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Tabla 17 



Correlations fetween the fflC Overall Skill b,h„s and th Dtensl.. »»Hm. Pr„iw 



the Jffictlvinaag Scale 



by thi Subjects on 



ikllli/DiiineionSj 



li Liaditshlp 



2, Organizing mi Planning 



3. Parci] 



on 

4. Decision Hiking 

5, Dieliivineii 
i. Intarpirsonil 



1, Adiptability 
8i Oral CoinynlcitlDn 
h Hrltten Ceiiiynication 
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Table 18 

Correlations Between thm MAC Overall Skill Ratines and the Dlroenslon 
Rating Provided by Each Rater Group on the Ch^erall RatlnR Item 
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Skills /Rater Groups s 


t 
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•"-1 
9 




Leadership 


.08 
(105) 


.36* 
C 73) 


-. 19* 
( 95) 


-.02 
(115) 


2. 


Organiglng and Planning 


.23* 
(105) 


.29* 
( 73) 


.02 

( 95) 


,07 
(115) 


3- 


Perception 


.23* 
(105) 


.23* 
( 73) 


. . in 
( 95) 


.09 

(115) 


4. 


Decision Making 


.26* 
(105) 


.16 
( 73) 


.02 
( 95) 


.10 
(115) 


D m 


Decisivaness 


. 14 
(105) 


* 19 
( 73) 


. 02 
(95) 


1 2 
(115) 




Interpersonal 


.22* 
(105) 


.14 
( 73) 


-.06 
( 95) 


.00 

(US) 


7. 


Adaptability 


.24* 
(105) 


.19* 
( 73) 


-.05 
( 95) 


.03 
(115) 


8, 


Oral Comoiunication 


.22* 
(105) 


.24* 
C 73) 


.02 
( 95) 


.00 

(115) 


9, 


Written Comiunlcation 


. 11 
(105) 


.20* 
( 73) 


.01 
( 95) 


-.01 
(115) 



Note. The for each cell appears in parenthasas. 
*S_< 0.05 
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based on the subordinatei ratings exhibit only a prepdnderance of positiva coef- 
ficients In the Job Performance Scale and the Ef f ectivenass Scale, While this 
situation suggests that a positive statistical relationship may in time be ex- 
hibited, at the present the validity correlations based on the subordinates' 
ratings exhibit no statistical relationship In all three instrunisnte (see Table 
11, 16 and 18), In the case of the subjects, the validity correlations based on 
this group's ratings exhibit likewise no statistical relationship in all three 
Instruments (see Table 12* 17 and 18)* Finally, a comparison between the re- 
sults of the MAC evaluation based on the Job Performance Scale and the results 
of the Sehmitt at al, (1983) study revealed that the two studies concur on the 
statistical relationship derived from the ratings provided by the two comparable 
rater groups, i.e., the supervisors and the subjects, FurthErmore, despite the 
greater Job tenure of the subjects in the Bchmitt study, the proportion of posi- 
tive and statistically significant validity correlations In the MAC evaluation 
compares favorably with the Bchmitt study (see Table 13)* 

Longitudinal View of the Validity Correlations 

Since the Job perfomance ratings of the original 47 subjects were obtained both 
in June of 1983 and June of 1984, it afforded an opportunity to view a portion 
of the validity correlations of the MAC longitudinally, Tha specific focus was 
on the previously noted tendency of validity correlations to rise over time. To 
determine If the K^C exhibits this tendency, the composite validity correlations 
for 1983 and 1984 were computed. The composite validity correlations are the 
correlations between the sum of the MAC overall skill ratings and the sum of the 
dimension ratings on each criterion instrument provided by each rater group. 
The increases in these correlations from 1983 to 1984 were tested for statisti- 
cal significance utilising a test of significance of the difference between cor- 
relation coefficients for two correlated samples. The test yielded no statisti- 
cal significance. 

Despite the lack of significance In the increases of the composite validity cor- 
relations, there was a discernible rise In the validity correlations of the MAC 
between 1983 and 1984. Tables 19 through 21 display the 1983 and 1984 correla- 
tions between the MAC overall skill ratings and the composite dimension ratings 
on the three criterion instruments, to ex^ination of these tables reveals that 
the positive coefficients across the tables rose from 55.5% in 1983 to 68.5% in 
1984, Consequently, there is some evidenca that the validity correlations of 
the MAC have risen over tijiie. 



Check for Criterion Contamination 



Since the MAC Is an oparational assessment center, the subjects* skill ratings 
were not kept completely confidential. Criterion contamination could have con- 
tributed to the positive statistical relationships exhibited by the validity 
correlations. To rule out this possibility, the evaluation incorporated a pro- 
cedure suggested by the Huck and Bray (1976) study as a check for criterion con- 
tamination. This procedure is essentially a comparison of the criterion ratings 
provided by the raters who were aware of the assessment center ratings with the 
criterion ratings provided by the raters who were unaware. The purpose of the 
comparison was to ascertain If the knowledge of the assessment center ratings 
biased the criterion ratings and produced inflated validity correlations. 

Of the four rater groups in the mC study, only the subjects and the supervisors 
had access to the MAC skill ratings. The subjects were routinely informed of 
their skill ratings as part of the assessment process. This knowledge, however, 
did not result In inflated validity correlations. Indeed, the validity corra* 
lationo based on the criterion ratings provided by the subjects exhibited no 
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Table 19 

Correlations Between the Overall Skill Ratings and the 1983 and 1984 

Compoiilte Ratings on the Job Performance Scale for the Original 47 
Subiects 
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Table 20 



Correlati ons Between the MAC Overall Skill Ratings and the 1983 and 1984 
Compoaite Ratlnga on the Effecti^enesa Scale for the Original 47 
Subjecta 
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Table 21 



CorrelatiQns Betweeri j =he Overall Skill Ratings and the 1983 and 1984 

Compoalte Ratings on t^-he derail Rating Item for the Original 47 
Subj acta 
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atatistiGal relationship (see Tablei 12, 17 and 18). ThuSi . the cheek for cri* 
terion contamination in this situation is not warranted. Fur rthermore , ±c is not 
possible, since the process of rciytlnaly informing all th^m subjects of their 
skill ratings precludas the application of the procedure , 

The check for criterion CDntamination, therefore, was confirr=iad to the supervis- 
ors. This group of raters was not routinely informed of the subjects' skill 
ratings, but they did have access to them by virtue of tb^ir positions within 
the school system. Consequently, prior to rating the subjac^ts* job performance, 
the supervisors were asked i "Are you aware of the skill ratings the [subject] 
received from the Management Assessiint Center?*' Of the 108 supervisors provid- 
lng_ criterion ratings, 42 respondid "yes" to this questioon and 66 responded 
"no". Based on these responses, the subjects were divided lt3ito two groups. One 
group consisted of the subjects rated by the 42 "aware" s^-^pervisors , and the 
other froup consisted of the subj acta rated by the 66 "unaws^^a" supervisors. 

The c parison of the two groups mi based on the data disp^ laved in Table 22* 
They include the means and standard davlations of the coop** osite overall skill 
ratings of the MAC, the means and standard deviations of the composite dimension 
ratings of each criterion instrutnint, and the validity c^^^relations based on 
these ratings. The differences betwiin the corresponding va^lues of the predic- 
tor and the criterion ratings were tiitad for significance utilizing analysis of 
variance. The test revealed that the differences are not statistically signifi- 
cant. This indicates that based on thi predictor ratings th^ two groups of sub- 
jects perf omed similarly at the HACp and this similarity Ip performance was re- 
flected in the subsequant criterion ratings* The fact that the criterion rat- 
ings provided by the aware supervliori did not differ sigai^icantly from those 
provided by the unaware is evidence of a lack of criterion co sntamlnation. 

Additional evidence is provided by thi coefficient values o£ the validity corre- 
lations * which are the reverse of what would be expected had there been criter- 
ion contamination. The validity corrilatlons based on tba^ dimension ratings 
provided by the unaware supervisors ira actually higher tha^ those based on the 
aware supervisors. Criterion contsfflination, therefore, coul^ not have inflated 
the validity correlations. To rule out the unlikely posslbllr.lty that it deflat- 
ed the validity correlations, the diffirences in the correa^pondlng coefficient 
values were tested for slgniflcanci utilising Fisher's Z t transformation in a 
test of significance between the correlation coefficients f =or two Independent 
samples. The test revealed that thi differences are not ata^tistlcally signifi- 
cant* Thus, the application of thi procedure suggested by the Huck and Bray 
(1976) study reveals no evidence of criterion eontamination the ratings pro- 

vided by the supervisors. 

Comparison of the Seliction Processes i Resul^^s 

The final phase of the MAC evaluation Involved a comparisoii of the district's 
present selection process for school-leval administrators ^it^a the former selec- 
tion process. The former selection process essentially consi^sted of a series of 
interviews for the qualified candldatiSi The present selectiion process differs 
in the use of the l^C to screen the qualified candidates p^^ior to the inter- 
views* The subjects In the comparlion consisted of 109 incimmbent school-level 
administrators. The Interview-only process was used to s^^lact 62 of them. 
These 62 subjects were hired during the 1981-82 school year* the last year the 
interview-only selection process vas operational. The rein^aining 47 subjects 
were hired the following school year using the MAC-interviai^ selection process. 
To equate the difference In job tenuri between the two grov^pSj the comparison 
was staggered In time. The June of 1983 Job performance rat-ings of the Inter- 
view-only subjects were compared to thi June of 1984 job per f-^ormance ratings of 
the l^C-intervlew subjects, 
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Table 22 

Check for Criterion Contamination 



Ratings and Correlations 



Subjects Ratad by I 

Aware Unaware 
Supervisors Supervlsori 
n ^ 42 n = 66 



Composite Overall Skill Rating of MAC 

Mean 42.4 44.8 

Standard Deviation 11,7 10.9 

Job Performance Scale Composite Rating 

Mean 63,2 66,2 

Standard Deviation 14.6 16-0 

Effectiveness Scale Composite Rating 

Maan 53,4 55^3 

Standard Deviation 10,9 11.6 

Overall Rating Item 

Mean 7.2 7,2 

Standard Deviation 1^6 1.8 

Correlation of the Composite Overall Skill 

Rating of MAC and the Composite Rating of 1 

Job Parformance Scale ,05 .11 

Effectiveness Scale ,08 ,24 

Overall Rating Item .19 ^24 
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Table 23 displays the job performance ratings for the two groups of subjects* as 
well as the results of the statistical comparison of these values utiligini an- 
alysis of variance (AN OVA) . Specifically, the second column in Table 23 dis- 
plays the means of the composite job performance ratings of the 62 interview- 
only subjects provided by the four rater groups on the three criterion instru= 
ments* The third column displays equivalent data for the 47 MAC-intervlew sub- 
jects* The fourth column displays the F value resulting from the statistical 
comparison of each set of means using AHOVA, The final column in the table dis- 
plays the significance of each value. 

A review of the second and third column in Table 23 reveals a pattern of con- 
sistently higher job performance ratings given to the HAC-interview subjects 
by the supervisors and support persons* This pattern is of interest, because, 
as noted in the second phase of the evaluations the job performance ratings 
providad by the supervisors and the support persons exhibit criterion-related 
validity* Nevertheless, the job performance ratings of the MAC-interview sub- 
jects are not of sufficient magnitude to preclude the possibility that their 
advantage over the ratings of the interview-only subjects resulted from pure 
chance. Indeed* an examination of the final column in Table 23 reveals that 
none of the differences in the means are statistically signficant at a probabil- 
ity value of p ^ 0*05* This indicates that based on the job performance rat- 
ings neither group of subjects exhi^^its a clear advantage over the other* 



The significance of the £ value refers to the probability that the difference 
between the two means was due to pure chance. 
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Table 23 



Comparison of the Job Perf ort aance Ratings of the Interyia-s^^Only Subjects^ and 
the M^C-Intarviaw Subjacts 



Hego CQmpoa ite Ratings ANOVA 



Supervisors' ' Interview-Only >^C-Intarvlew Sign— flcanee 

Ratings on the* Subjec^l Subjects F 



0=^ F 



Job Performance Scale 7-52 7.81 

(56) (43) 1.097 ^-297 

Ef f aetiveness Scale 65*80 70,33 

(56) (43) 2 . 204 ^141 

Chferall Rating Item 56,73 58,95 

(56) (43) 1,269 263 



Support Persons' 
Ratings on the: 



Job Performance Scale 7.48 7.48 

(^2) (29) *000 *.^986 

Effectiveness Scale 65,90 67,83 

C42) (29) ^190 ,-^664 

CKrerall Rating Item 54»8l 57.38 

(42) (29) .680 , , 412 



Subordinates ' 
Ratings on the I 



Job Performance Scale 
Effectiveness Scale 
Overall Rating Item 



8.31 

(55) 
76.02 

(55) 
62,73 

(55) 



8,25 

(44) 
75,39 

(44) 
63,27 

(44) 



143 
054 
165 



706 
816 
685 



Subj ects ' 
Ratings on thei 













Job Performance Scale 
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.T706 
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.^16 
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63.27 








(55) 


(44) 
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.0685 



Notes The n for the cell appeati in parentheses. 



CONCLUSIONS 



The conclusloni o€ tk a valuations of the Management Assessment Center (MAC) 
will be addresiad 4^ this section of the report. It should be noted that this 
section adherei to chi same format employed in the Design of the Evaluation and 
the Results of the Mua^lon, in that the conclusions of the three phases of 
the evaluation are address «d separately* 



Intervt^i^ and Survey of the HAC Assessors i Conclusions 

Since the intarvl#vr and th« survey of the MAC assessors were conducted after the 
completion of Bmm&^mmt =ycle 1 in 1981-82, the conclusions of ^-hii phase of 
the evaluation '^^tm praieated in the preliminary report which wa^i published in 
March of 1984, Basically the preliminary report noted that during the first 
year of operation* thi tLa.C had experienced some start-up probiems. The MAC 
staff, however, had Imn v«ry responsive in addressing these problems , and thus 
had facilitated th^ eubse^uant development of the MAC. Consequently, the MAC 
assessors, who wmt^ in a unique position to observe the operation of the center, 
were very support iv^ of both the mC staff and the MAC process. Beyond thesa 
general conclusion^ , tha p=-eliminary report presented a number of ancillary con- 
clusions which will not b« replicated in this report. A reader who wishes to 
review all the QonQlualon^ of this phase of the evaluation should contact the 
Office of Education^ jAccoaritablllty and request a copy of Preliminary Report an 
the Evaluation of gtyg Hana^ament Assessment Center ^ ~ ~ 



Crttearion-ReLated Validity of the ^Ci Conclusions 

The second phase ot the evaluation involved the criterion-related validity of 
the MAC, The design of thLs phase encompassed several data analysis procedures ; 
they included: (a) ciii int areorrelations of the predictor, (b) the intercorre- 
latlons of the atit^^rlon, Cc) the inter-rater reliability of the predictor, (d) 
the consistency of ttii erife arion, (e) the validity correlations, (f) the longi- 
tudinal view of thi ^alldicy correlations,, and (g) the check for criterion con- 
tamination. Tha re^tjlta of these data analysis procedures yielded the following 
conclusions I 

o A degree of c^giunal^ty Is revealed by the intercorrelation of the nine 
MAC skill rat itip. E^is coTOunallty Implies that some of the skilli are 
tapping a ca^ipn fsftor in the subject's performance. This situation* 
however, is tiO^ unusu^al given the similarity of some of the skills. In- 
deed, a modera^i lev« J. of intercorrelation under the circumstances is to 
be expected , overriding consideration Is whether the intercorrala- 

tlons are s© UtjiformLy high as to suggest that some of the skill ratings 
are redundant* A fac -eor analysis revealed that this is not the case with 
the MAC skill tatlngs* Although they exhibit a degree of conmunality* 
each skill li ^ssint^ally providing unique information on the iubject's 
performance % 

Q As with the MAC akill ratings, uniformly high intercorrelations of tha job 
performance diMn^lon ratings may denote a co^unallty across the dimen- 
sions. A moifa iikely interpretation, however, is that it denotes thi ex-- 
istence of a h^jo effect. A halo effect occurs when a high rating on one 
dimension lnflti#neii ^he rater to give the subject comparable ratingi on 
the other dlBia^iloni* Since the job performance raters (unlike the MAC 
assessors) Wert liot trained in observing and rating a subject 's perform- 
ance, they coald con&aivably make such a mistake* An examination of the 
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i^ntercorrelatlons of the dimensions revaals that coefficients range from 
^^ow to modarata-tp-high, but they are never so high as to suggest that each 
Q^lmension rating is not providing unique information on the subjact'a job 
^performance* 

Q The most important measure of the reliability of an assessment center is 
the intar-rater reliability, which Is an Indax of the degree of agreement 
among tha assessors on the ratings of the subjects. This index is impor= 
tant, becausa in an assessment canter high intar-ratar rallability is con- 
sidared a prerequisite to validity. The intar-rater reliability of the MAG 
-T^as computed using two methods, tha Pearson £ and coefficient alpha* Both 
^methods yielded a high index of inter-ratar reliability, indicating a high 
-degree of agraament among the MAC assessors, Dapending on tha mathod used 
-^o compute tha index* the inter-rater raliability of the MAC assessors is 
•-either equal to or higher than the inter-rater reliability of the NASSP 
.^^ssessmant center assessors reported in the Schmitt et al, (1983) study* 

0 — i^he correlation of the composite job performance ratings on each criterion 
^instrument provides a measure of the consistency of the ratings* These 
c^orrelations of consistency yielded comparably higher coefficients for the 
^ob performance ratings provided by the supervisors and the support per- 
sons than the subordinates and the subjects. Nevertheless * on the whole 
-tthe correlations of consistency were high, which reflects favorably on the 
^reliability of the three criterion instruments. 

D ^The positive statistical relationships exhibited by the validity correla- 
te ions indicates that the MAC did to a degree predict the subsequent job 
^performance of the subjects. Furthermore, a comparison of the results of 
tehe MAC evaluation with the Schmitt et al, (.1983) study reveals that the 
E^roportion of positive and statistically significant validity correlations 
^are comparable. This outcome is impressive, because the tendency of the 
%^alues of validity correlations coef f icents to rise over time and the 
^^horter job tenure of the MAC subjects should have combined to yield com- 
^^arably lower validity correlations for the MAC, 

Q ^K'wo sets of validity correlations were computed for the original 47 sub- 
J acts using the job performance ratings obtained in June of 1983 and June 
1984* A comparison of these two sets of validity correlations reveals 
&^ discernible rise in the values of the coefficients ever the one year 
s ^pan* While the difference in values is not statistically significant, it 
A^oes concur with several studies that have noted this tendency of the 
v^^-^lues of the validity correlations coefficients to rise over time* 

0 S. ince the MAC is an operational assessment center, the subjects' skill 
r^^tlngs were not kept completely confidential* If the criterion raters 
w^*^re aware of the subjects* MAC skill ratings, it could have contributed 
tttO the positive statistical relationship exhibited by the validity cor- 
r relations. Of the four groups of criterion raters, only the supervisors 
a^ttd the subjects themselves had access to the MAC skill ratings. The sub- 
J^ects were routinely informed of their ratings during the assessment pro- 
c^ess. However, the validity correlations based on the job performance 
^^atings provided by this group exhibited no statistical i elationship , so 
c^^lterion contamination was not an Issue, The supervisors * although not 
rc=iutlnely informed of the MAC skill ratings, had access to them by virtue 
o^^E zhmtT positions within the school system. Consequently, the check for 
c^^iterion contamination suggested by the Huck and Bray (1976) study was 
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applied to this group* It yielded no avidenca of criterion contamination. 
Thus, having knowledge of the subjects* MAC skill ratings did not bias the 
supervisors when they rated the job performance of the subjects. 



Comparison of the Selection Processes i Conclusions 

The final phase of the MAC evaluation involved the comparison of the district's 
present selection process for school-level administrators with the former selec- 
tion process. The former selection process essentially consisted of a series of 
Interviews for the qualified candidates. The present selection process differs 
in the use of the MAC to screen the qualified candidates prior to the inter- 
views. The results of the comparison of the two selection processes yielded the 
following conclusions I 

o The comparison of the job performance ratings of the MAC-intervlew sub- 
jects and the interview-only subjects revealed that the !ttC-interview pro- 
cess is not superior to the interview-only process in the selection of 
better school-level administrators. Despite the validity of the MAC, its 
incorporation into the selection process is not justified by the selection 
of batter candidates. Thus, it must be concluded that under the existing 
operating procedures the MAC has no utility. 

o The unsatisfactory perfomance of the MAC-interview selection process can 
be attributed to two possibilities. The first possibility is that the MAC 
and the district's interview procedure have comparable validity, Canse- 
quently, nothing was gained by Incorporating the MAC into the selection 
process. This possibility is highly unlikely , since research has shown 
that the interview method only eKhibits validity correlations coefficients 
in the 0.10's (Reilly I. Chao, 1982). The MC's current validity correla- 
tion coefficients based on the ratings of the supervisors and support per- 
sons are in the 0,20'Sp and there is evidence that they are still rising, 
A more likely possibility is that, with the existing high passing rate, 
the MAC results are not really being used. In other words, the minimum 
passing score of the MAC Is such that the few candidates who are elimin- 
ated from consideration would probably have been eliminated an^ay by the 
interviews. Under the circumstances, the interviews in effect became the 
overriding factor In both selection processes. Thus, there was no advan- 
tage in incorporating the ^C into the selection process, not because of a 
deficiency in its validity, but because its validity was essentially not 
used . 

In brief, the evaluation of the MAC revealed that -er a job tenure of two 
years or less, the correlations between the subjects* MAC skill ratings and the 
job performance ratings exhibited criterion-related validity. Furthermore, the 
longitudinal trend of the validity correlations, as well as the high inter-rater 
reliability of the MAC assessors, are indications that the values of validity 
correlations coefficients will continue to rise. How high they will go can only 
be speculated at this time. However, if the research in the field is an indica- 
tion, the validity correlation coefficients should be in the low 0,40's when the 
subjects achieve a job tenure of approximately five years. Nevertheless, the 
MAC^s level of validity in the future or in the present are moot issues, if the 



IJtility refers to "the determination of expected institutional gain or loss 
(outcomes) anticipated from various courses of actions "(Cascio, 1^'82). The 
fundamental question of utility analysis iss Are the costs of a course of 
action justified by the returns? 




passing rate remains so high as to praclude the center's utllicy. The impor- 
tance of this issue cannot be overstated. Perhaps Dr. Neal Schmitt phrased it 
best when he made the follQwing comment about the I^iACi 

Your pass rate* as well as those of other centers * is really arbitrary and 
can be manipulated* Actually, th i optimal use of the assessment center 
data is to pick the highest ranking individuals. Use of [a high] suc- 

cess rate is tantamount to ignoring the results of the [^mC] (N. Schmitt, 
personal communication * March 23, 1984). 



13 

Dr. Neal Schmitt, who headed the validity study of the NASSP assessment cen- 
ters, served as a consultant in the evaluation of the MAC. 
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RECOM4ENDATION0 



The conciuaions drawn from the evaluation of the UAC serve as a basis for the 
following recommendations • 

o The MAC will not have utility until the minimum passing score is raised. 
It is J therefore, reco^ended that a qualified consultant be retained to 
advise the district on this matter. The employment of a consultant is 
recommended, because a higher minimum passing score will likely increase 
the adverse impact of the ^^.C* Consequently, it would be prudent to In- 
sure that all actions taken regarding this matter are legally defensible. 

The raising of the minimum passing score of the MAC is the overriding recomiend- 
at ion of this evaluation. The remaining reconmendations are all contingent on 
the Implementation of this recoimendation* 

Q Assuming an appropriate upward adjustment of the minimum passing scores it 
is reeoimended that the district retain the MAC as a part of its selection 
process of school-level administrators. This reeomendation is based on 
the established criterion-related validity of the l^C, as well as the dem- 
onstrated competence of the MAC staff. The MAC clearly offers the dis- 
trict a method of improving the effectiveness of the existing selection 
process* 

o The eKlsting procedure of the imc does not provide the data needed to com- 
pute the inter-rater reliability of the assessors. As previously noted, 
the index of inter-rater reliability is the most Important measure of the 
reliability of an assessment center. This index is useful in monitoring 
the training and performance of the assessors. For this reason, the B^C 
procedure was temporarily altered to provide the evaluation with this in- 
dex. And, despite the fact that the evaluation revealed the index to be 
high at this time, this situation could change. Therefore, it is sug- 
gested that provisions be made to compute the inter-rater reliability in- 
dex of the MAC whenever there is a substantive change in the training pro- 
cedure or in the cadre of assessors. 
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M/R RATING INSTRUMENT 




2-4 



INCUMBENT/RATER CODE 



DADE COUNTY PUBLIC SCHOOLS 
OFFICE OF EDUCATIONAL ACCOUNTABILITY 



SECTION A: RATER DATA 
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RATER DATA 



Instructions i All raters shduld raspond to itms 1 through 7« 

1, Rater's agaj _____ ysara 

2, Sex I femalo male 



3. 



1 2 

Ethnicity: 
White, not of Hispanic origin 
Blacky not of Hispanic origin 
Hispanic 

Asian or Pacific Islande^r 
American Indian or Alaskan native 

Current Work Location^ 

Elamentary School 
Junior High School 

Senior High School 

Area Office 
Central Offica 



Current posit ion i 
Teacher 

Asst^ Principal 
Principal 
Director 
Otheri 

(Please specify*) 



6, Nmnbar of years with the Dada County Public Schools: 



yean 



7. Are you rating yourself (i*e., the rater and the incumbent are 
the srae person)? yes no 



DO NOT 
WRITE IN 
THIS AREA 

6 = 7 



10 



11 



12-13 



14 
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Instructions I Item 8 is only applieable to inctnabents coded M (see 
box 1 on ¥he eover) » Only a rater 1.^0 is currently a principal or 
en area direetor shou 1 J r i spond ^ o it to 8 a A pr Inc 1 pal who is r a t^ 
ing him/herBelfV however, should not raspond to this item, 

8, Are you awara of skill ratings the incmnbent received from the 
Hanagament Assessment Center? yes no 

Instruct ions ; If you are not rating yourself^ skip itras 9 through 
12 and go dlraetly to page 5= If you a- 1 rating yourself * please 
respond to itras 9 through 12 # ^ 

9, Were you evaluated at the Management Assessment Center as part of 
the selection procedure for your current job? yes^ no 

"1~ 

10, How many months have you held your current Job? (include summer 
months regardless of employment status. Include time served on 
an "interim" basis ^ if applicable.) month/s 

11, ^s^at was the last job you held prior to assuming your current 
Job? 



Teacher 

Department Head 
C.SpIp Director 
Activities Director 
Counselor 

Assistant to the Principal 
Assistant Principal 

Assistant Principal at a different school level 
Principal at a different school level 
Other; (Please specify,) 



3 



10 



12, How many years did you hold your previous job identified in 
item 11? year/s 



DO NOT 
WRITE IN 
THIS ARIA 



15 



16 



17-18 



19-^20 



21 - 22 



EKLC 
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SECTION B: JOB PERFORMANCE SCALE 
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JOJ^ PERFORMANCE SCALE 



InstruGtions >Thig rating seale ia designed to measure the impprtant 
dmeneions of a principal's or assistant principal's job. You are to 
use the scale to evaluate the job perforraance of a designated incm- 
bent. Eaeh dimension of the scale should be considered separately 
and carefully, as some incumbents may do well on one dimension but 
not another. These ratings may include some areas of the Job which 
you do not see the incumbent doing or for which you do not have 
enough information to base a decision. In this situations feel free 
to indicate you are unable to rate the dimension in question. 

Each dimension has a rating scale which is numbered from 1 (the low- 
est rating) to 6 (the highest rating) , ExTOples of job performance 
behavior assoQiLated with various nmnerical ratings are provided. 
These are actual examples obtained from teacheraj students j princi- 
pals^ support staff, and parents in a national study of school prin- 
cipals and assistant principals* To make a particular rating does 
not imply that the incinnbent actually does or has done the things 
described by that rating . The exmples are used to illustrate the 
kinds of behavior that can be expected from an incimbent who is typi- 
cally rated highj average or low on each dimension. Thus ^ the exam- 
ples represent a point of reference to help you decide how to rate 
the incumbent. 

The actual rating procedure will be to read each dimension and the 
examples of job performance behaviors. If you feel that you have in= 
sufficient information to rate the dimension, check the box at the 
top, right-hand side. Otherwise decide which numerical rating best 
reflects the incumbent's job performance^ and check the appropriate 
box on the right. Note that there is only one dimension per page, so 
CHECK ONLY ONE BOX PER PAGE . 
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1* CURRICULUM AND INSTRUCTIONAL LEADERSHIP j 
MONITORING CURRICULUM OBJECTIVES 



CHECK ONLY ONE 
RESPONSE 



If you have Insufflelent information to rata this dimanslonp cheek 
this boXb 



Examples of Bahaviof of Individuals who are Usually Rated "High" on 
this Dimension s - 

^ Takes a stand on issuas even when thay are controvera ial ; supports 
and itands up. for educational values* 

^ Constructs a schedule which maKimizas student options j yet mini— 
migas course conflicts ^ 

^ Visits classrooms to mpnitor the curricul™ actually being taught 
in school. 



Examplas of Behavior of Individuals \^o are Usually Ratad "Avarage" 
ot^ this Dimension s - - - - 

- Raads necessary materials to have a thorough knowladga of stata or 
district eurriculma standards and requirements . 

Looks at new texts so that he/she can influance the selection of 
raatarials and textbooks, 

- Raviaws and makas comments on weekly lasson plans in terms of over- 
all eurricul™ objactivas* 
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a 

RATIHG 

6 Q High 



5 □ 



^ d 



Averag 



Examples of Behavior of Individuals who ara Usually Rated "Low" on 
th is Dimens ion i ~^ 2 

^ Does not utilize special curriculum programs providad by tha dis- 
trict. 

^ Adopts curriculum materials without tha input of teachers or in 
spite of thair recommandations * 

- Does not allow any curricular axparimentation by faculty members, I Q Low 



DO NOT 
WRITE IN 
THIS AREA 
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CURRICULUM AND INSTRUCTIONAL LEADERSHIP; 
MONITORING INDIVIDUAL PROGRESS 



CHECK ONLY ONE 
RESPONSE 



If 7QU hava issufficient information to rata this diaengion, chaek 
this box* 



Examplas of Behavior of Individuals who are Usually Rated " High" en 
this Dimension : " " " ----- 

- Initiates a program with the help of student-parent associations to 
allow students to take advanced classes offered at nearby schools* 

- Organizes student help aessions which meet after school hours for 
students who are failing* 



Examples of Behavior of Individuals ^o are Usually Rated "Average' 
on this Dimansioni 



- Reviews records of individual stadent progress inQluding standard- 
ized scores and basic skills program* 

= Institutes a classroom where several things fire taught (e^g*, cer~ 
amies, painting) because of a lack of enough interest in any one 
of them to justify individual classes* 

- Sponsors a biweekly reading rally in which all students stop what 
thay are doing and read for 20 minutes* 



Examples of Behavior of Individuals who are Usually Rated "Low" on 
th is D imens ion : ~ 

^ Allows students to hand carry scores of certain tests home; stu- 
dents are able to open these unsealed envelopes and compare scores* 

" Schedules all advanced courses in the morning so that academic stu- 
dents can take only 2 out of 5. 
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a 

RATING 

6 Q High 



Averag! 



Low 



DO NOT 
TOITE IN 
THIS AREA 
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3* COORDINATION OF STUDENT ACTIVITIES^ SUPERVISION 



CHECK ONLY ONE 
RESPONSE 



If you hava insuf f ieiaiit tnforaation to rata this dlnansio 
this boxt 



ehack 

RATING 



Examples of Bahavior of Individuals who are Usually Rated "High" on 6 High 
this Dimanaipm ^ " 



^ Initiates an awards banquat and astablishes a lettar for superior 
aoadamic performaneai 

^ Evaluates all activities and student needs with input from faculty^ 
student oouncil, and studant survtys, than acts on results. 5 



- Meets regularly with studant leaders to eoordinate activities and 
take suggestions. 

— Reschedules a homecoming avent which had been scheduled for the 
night before tha SAT, 



Examples of Behavior of Individuals ^o are Usually Rated "Average" 
on this Dimansion i ^ — 

" Schedulas a sporting fivant during the aeadamic day for the benefit 
of bused and/or poor stvdenta who cannot coma to night activities. 

~ Organizes parent^studant taams to work on fund raising activities , 

- Organises faculty teams to play againsc varsity teams. 



Examples of Behavior of Individuals ^lo are Usually Rated "Low on 
this Dimension: 



Average 



3 □ 



^ Organizas a student fund raising activity without talking with stu- 
dents, 

- Cancels a pep meeting because of a drinking party by a few stu- 
dents, 

^ Allows extra-curricular activity and sehool classes to conflict so 
that students have to be pulled out of class to participate in band 
or sports. 



a 



Low 



■75 



DO NOT 
WRITE IN 
THIS AREA 
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STUDENT ACTIVITIES.- PARTICIPATION 



CHECK ONLY ONE 

' "RESPONSE 



If you have Insufflciane information to rate this dmensionj eheck 
this box» 



Examples of Bahavior of Individuals who are Usually Rated " High" on 
t u i s D imens xQ a : ~ ~ -- 

- Initiates a program so that handicappad students are ineorporated 
into various activities, such as plays or sports* 

^ Participates in extracurricular school activity by actually working 
at the function^ such as fun fairj school dinners. 

" Provides student government officers with the opportunity to so= 
licit student participation* 



Examples of Behavior of Individuals wh^ are Usually Rated "Av erage' 
on this Dimension* - - 



^ Talks to students who are not participating in extracurricular act- 
ivities to get thra involved* 

^ Attends extracurricular activities, 

-Encourages staff to participate in extracurricular activities but 
does not demand such participation. 



Examples of Behavior of Individuals \dio are Usually Rated " Low" on 
this Dimension .' 

-Tells the students that certain activities are important and then 
doesn't support them by his/her presence or allowing time in the 
schedule for thOT, 

- Selectively participates in only certain school activities (e.g., 
foot ball) , 
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RATING 

6 □ High 



Averagi 



2 □ 



1 Q Low 



DO NOT 
WRITE IN 
THIS AREA 
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5. DIRECTION OF SUPPORT SERVICES 



check only one 

' response 



If you have losufficienC Infarmatian to rata this dimansioni oheck 
this boK« 



□ 

RATING 



ERIC 



Examples of Bahavior of Individuals are Usually Rated "Average" 
on this Dipaansion : 

^ Initiates a program to have parants receive first aid training so 

there ean ba personnel in the clinic during the lunch hour. 

- Organises groups of support services people so that they have a 
more unified voice* 

- Asks cafeteria and custodial staff for input in school decisions. 

^ Asks taachers to periodically coinplete a form concerning the qual= 
ity of support services as an aid in monitoring these activities. 



Examples of Behavior of Individuals who are Usually Rated "Low" on 
this Dimensions 



^ Assigns hall and cafeteria duty without staff input* 

Allows a co^unity group to use a school projector without the per^ 
miasion of the madia specialists* 

Does not consult with custodians when making requisitions for sup= 
plies • 

^ Neglects to inform secretary of the school programs and events. 
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Examples of Behavior of Individuals who are Usually Rated "High" o n 
this mens ion : 

^ Devises a system whereby librariaT is able to coordinate library 
resources (e*g.s books, magazines) with needs of teachers as indi- 
cated by If^qson plans. 

Includaf ,arical/custodial staff in all school parties and meet- 5 |~) 
ings* 

- Verball;' acknowledges the completion of tasks by school maintenance 
and food services personnel. 



High 



Averag 



2 □ 



Low 



DO NOT 

TOITE IN 
THIS AREA 
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6. SUPPORT SERVICES I DIRECTING THE BEHAVIOR OF STUDENTS 



CHECK ONLY ONI 

RESPdNSE" 



If you have insuffielenc information to rate this dimansioni cheek 
this boK^ 

Examplas of Behavior of Individualg who are Usually Rated "High'- on 
this Diffiang ion i " 

* Maintains up-tQ^date staff manuals including statements on disci- 
pline policies j which conmiunieate all procedural matters, 

" Works with teachers and students to handle discipline problems* 

^ Helps counselor develop programs for incoming students , such as 
orientation. 

Sets guidelines for student behavior which are not threatening and 
enforces them objectively. 

- Conducts a cafeteria survey to determine student needs in that area. 



EKamples of Behavior of Individuals ^o are Usually Rated "Average" 
on this Dimension : ~ ~ 

^ Develops a staggered release schedule on rainy days to alleviate 
traffic jams in front of school caused by parents picking up their 
children, 

- Develops special instructional progrfflns for lunch period so that 
students do not just play in g3?m or hang around, 

- Starts a program in which highest attending elassroom is recognised 
with a free period and a party. 



a 

RATING 

6 Q Higli 



5 a 



4 □ 



Averaj 



3 d 



Examples of Behavior of Individuals who are Usually Rated "Low" on 
this Dimension ; - _ _— _ _ 

2 

- Discontinues an arts program because a small group of students 
ruined a kiln. 

- Suspends a student with whom the counseling staff is working close- 
ly; does not check with staff first. 



Does not discipline students because he/she does not know whac to 
do or does not have time. 

Places a student in a special education program for reading prob- 
lems without proper evaluation. 



Low 



11 



78 



DO NOT 
WRITE IN 
THIS AREA 
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7, STAFF EVALUATION 



CHECK ONLY ONE 
RESPONSf 



If 7©u have insufficient informacian C© rate this dimension, eheek 
this bo%* 



Examples of Behavior of Individuala who are Usually Rated "High" on 
this DlmensLoni — _ _ _ 

^ Provides eonstructive feedback after obsarvations and calls in spa- 
eialists if additional help is needed; then obsarves again and gives 
more feedback, 

^ Encourages evaluation of hm/hersalf by staff* 

^ Consults with individual staff meimbers on a pariodic basis to de^ 
valop individual standards of performance (goals and objectives) 
and reviews subsequent accompl istaent of goals. 

- Writes a note to each teacher who has led a particular event or 
done something special to show appreciation and give praise* 



Examples of Behavior of Individuals who are Usually Rated "A verage" 
on this Dimension: 



" Disciplines employee who is not functioning properly. 
^ Asks permission to come to observe classroom « 



Examples of Behavior of Individuals who are Usually Rated "Low" on 
this Dimension : 

~ Doesn't afford a teacher adequate opportunity to refute accusations 
made by students, 

^ Circulates a staff bulletin criticizing teachers whan only 3 or 4 
teachers were guilty and should be reprimanded individually, 

" Criticizes or belittles staff members in front of other staff mem- 
bers, students or parents. 



□ 

RATING 



High 



12 



79 



5 a 



Average 



□ 



Low 



DO NOT 
WRITE IN 
THIS AMA 
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8. DEVELOPMENTAL ACTIVITIES 



check only one 

response' " 



If you have insufficient inforaation to rate this dimensianj ehaek 
this boxe 



= Davisas class achedula to allow for staff devalopment periodi. 

^ Uses staff meetings to ancouraga and publiQiEa in-servica training, 

- Allows teachers time off to attand meetings and classes* 



Examples of Behayior of Individuals who are Usually Rat ed "Low'* on 
this Dimensions ^ 

- Talis staff to gat involved in staff developmant problems whan he/ 
she does not get involved, 

- Plans staff development activities with no staff input. 

^ Does not help teachers who have problems but works toward docuraant- 
ing problems and removing teacher, 

- Does not allow staff to go to professional meetings even at their 
own expense* 
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□ 

RATING 



Examples of Behavior of Individuals who are Usually Rated "High" o n 6 
this Dimension ; — 

- Provides in-service programs fur staff which include dealing with 
student behavior problems and interactions with parents, 

= Attends seminars * workshops ^ etc, for professional growth, 

" Provides time for teachers who have attended developmental activi- 5 Q 
ties to communicate information to the rest of thtt staff. 



Examples of Behavior of Individuals who are Usually Rated "Ave rage' 
on this Dimension: — -- 



High 



Average 



1 □ 



Low 



DO NOT 
WRITE IN 
THIS AREA 
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9* CObMJNITY RELATIONS 



CHECK ONLY ONE 
" RESPONSE 

If yeti hava insuf fie lent information to rata this dimansionp ehaek ~ 
this box. ^'"^ 

RATING 

- - _ . . 6 □ High 

Examples of Behavior of Individuals who ara Usually Rated "High" on 
t h i B D imen m^on i - - — ^ _ 

~ Inforras eoittmunity about tha basics of oparating the school and edu-- 
cational object ivas| asks for the ideas, opinions and support of 
these peopla and listens to their suggestions. 

^ Works with various community and loeal groups to develop coopera= 5 
tion with the school * 

= Institutes a GOurse which is aimed at developing cultural awareness 
in an integrated community. 



EKamplas of Behavior of Individual^ who are Usually Rated "Average" 
on this Dimension i 

^ Sands stories to the coimunity newspaper and/or invites newspaper 
correspondents to school events • 

" Organizes senior citizen breakfasts to explain school programs* 

- Coordinates school activities with church and community events, 

" Takes two students to the Rotary Club every month as a reward and 
to promote good relations* 



Examples of Behavior of Individuals who are Usually Rated "Low" on 
this Dimension ; 

Stays away from comiunity organizations such as Jaycees, Lions , 2 
Kiwanls, etc* 

^ Defends his/her position to the coimnunity rather than listening t 
their requests for certain programs* 

Speaks in educational jargon to explain school programs to cormnun 
ity memberfi , 



Speaks badly of the school system to the news madia. 



DO NOT 
WRITE IN 
THIS AREA 



Average 



3 



Lew 
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10* INTERPERSONAL EFFECTIVENESS 



CHECK ONLY ONE 
RESPONSE ~ 



If you hava inpufflcient InformaCiDn to rata this dimansioni ehaek 
h is box • 



■ Kdmplas of Behaviof of Individuals era Usually Rated "Kigh" on 

this DimensiQii .* 

~ Shows a sense of himor in times of conflict, 

- Intaracts with students during tha lunch hour in tha cafataria* 

- Encourages parents to go directly to a teacher with a coraplimant 
instaad of ralaying the messaga him/harsalf to tha taacher. 

- Displays studants' art work in main office as ancouragement . 

= Comes from bahind desk to discuss problems, takes notaSj and rafars 
to this fila latar whan another discussion takes place. 



Exampl ae of Behavior of Individuals ^o are Usually Ratad "Avaraga" 
on this Dimansion: 

= Leaves a nota of anoouragamant to someone who has mada a mistaka, 
to carry on and start ovar, 

= Informs staff of all contributions to school activitias by faculty 
members and communicatas appreciation, 

- Allows staff member to go home during tha day to solve a family 
problem. 




High 



Average 



ExTOples^of Bahavior of Individuals who are Usually Ratad "Low" on 
this Dimensions 

^ Dalagatas work but than takes work away or intarfaras because he/ 
she thinks it is not getting done corractly. 

Uses languaga and words which students do not undar stand t^en com- 
municating with them. 

— Establishes a policy without giving staff or students a rationale. 

— Gats into "shout— outs" with students in tha hallways* 



1 □ i-ow 



ERIC 



15 



82 



DO NOT 
WRITE IN 
THIS AREA 

32 



11, COMMUNITY RELATIONS" PARENTS 



CHECK ONLY ONE 
RESPONSE 



If you have insufficient inforaation to rata this dimension, cheek 
this bo^e 



a 

RATING 



Examplas of Bahavior off Individuals who are Usually Rated "High" on 6 pi High 
this DimansTon : " — 

^ Starts a proeedura whereby parents coma to meet teachers individ- 
ually to discuss child's prograas* 



- Organizap coffaas in parent a ' homes to interact with parents in a 
nonschool environment* 

- Writes a letter to all parents inviting them to schools spends an 
eveniiig talking with them and answering questions. 



Examples of Behavior of Individuals who are Usually Rated "Average* 
on this Dimension: ~ 



Lements a program in which parents follow class schedule of their 
children for a shortened day, 

^ Organizes a system for buses to pick up parents and bring them to 
school events in order to promote coBmunity involvament. 

^ Galls parents to remind them of parent^teacher conference so as to 
ensure attendance* 



5 a 



Average 



Examples of Behavior of Individuals ^o are Usuall y Rated "Low" on 
this Dimension : ^ 

Refuses to develop or initiate a parent advisory group, 

^ Controls PTA by dominating officers, promoting participation of 
only one segment of school population^ or failing to follow through 
on their requests or suggestions, 

- Does not have a system of reporting students' discipline problems 
to parents . 

- Tells parents they do not really know their children when problems 
arise. 



Low 



DO NOT 
WRITE IN 
THIS AREA 
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12* COORDINATION WITH DISTRICT AND OTHER SCHOOLS 



check only one 

re'sp5nse"" " 



If you hava insuffleient information to rate this dimension, check 
this hmm 



Examples of^ Bahavior of Individuals who are Usually Rated "High" on 

this Diuiension i ~ 

— Participates in professional prganiEation problera solving projects 
aimed at improving the functioning of central administration serv=- 
ices which impact directly on the sehool, 

= Balances district activities against building priorities, 

— Encourages use of district resource personnel to develop needed 
programs * 

— Informs central office of possible complaints or problems at school 
or in comraunity so they will ba aware of it ahead of time. 



Examples of Bahavior of Individuals who are Usually Rated "Avaraga" 
on this D imen sion i 

^ Arranges meetings with central office administration to speak to 
staff at school on various issues* 

- Implements decisions at the building Tfhich were made at the dis- 
trict level even if not involved in making decisions, 

- Participates in district level eommittaas whan called upon. 



17 
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□ 

RATING 

6 Q High 



Average 



3 n 



EKamplas of Behavior of Individuals ^o are Usually Rated "Low" on 
this Dimension s 

- Makes all decisions without the advice of any imediate superior, 2 

^ Is so involved in district comiittees that he/she is always out of 
the building, 

- Does not coimunicate with district paraonnel unless absolutely nec^ 
essary , 

- Never volunteers for district committees, 1 



Low 



DO NOT 
WRITE IN 
THIS AREA 
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13, FISCAL OR MONETARY MANAGEMENT 



CHECK ONLY ONE 
ffiSPONSE 



If you hava insuffieient infarmation to rate this dimansion^ eheek 
this box^ 



Examplea of Bahavior of Individuale who are Usually Rated "High" on 
this D imen s ion i 

- Involves all staff in astablishinf priorities for the allocation of 
resources and materials, 

" Periodically reviews and shifts alloeations based on currant needs; 
obtains agraamant from all dapartraent haads* 

^ Haintains strict requiramants on tha prpceduras to follow with pur^ 
chase orders and budgets of departments. 

^ Acquaints tha staff with the daily oparations budget and how it re^ 
lates to their curriculum. 

Calls several other schools and sources to get materials when they 
are not available at the local school* 



EKamplea of Behavior of Individuals who are Usually Rated "Average" 
on this Dimension : 

- Monitors expendituras of students' activity funds. 

^ Provides funds for special art projects so that students would not 
nead to bring materials from home. 

^ Kaeps staff informed of budget. 



jExamples of Behavior of Individuals ^o^ are Usually Rated "Low" on 
this Dimension: ~ 

Does not comply with standard accounting procedural* 

- Misspends school money in one area and compensates by taking money 
out of budget in another area. 

- Spends an inordinate amount of school funds on athletics. 

- Requires staff to fill in .requisition forms for small items s such 
as toilet paper and light bulbs. 
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□ 

RATING 

6 Q High 



Average 



2 d 



1 Q Low 

DO NOT 
WRITE IN 
THIS ARIA 
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14* MAINTENANCE OF SCHOOL PLANT 



CHECK ONLY ONE 
RESPONSE 



If 70U have insuffieient information to rate this dimanaionp ehaek 
this boXe 



Examplag of BehaviQr of Individuala who are Usually Rated "High" on 
this Dimenaion : " ~"~ 

^ Works with new students to davalop school prida and reduca vanda= 
lisra. Older studants ara givan responsibility for taking new stu= 
dents "under thair wing", 

- Initiates a program to elaan up graffiti in school; provides stu- 
dants with cleaning materials and develops a contest for claanast 
araa, thereby unifying studants and staff, 

- Establishes a process which allows stafjf and community mambars to 
provide input for the orderly improvement of school plant facili- 
ties and equipmant . 



Examplas of Behavior of Individuals who ara Usually Rated " Avar age' 
on this Dimansioni " 



- Takes an activa part in kaaping the building claan Ce,g,j picking 
up tha paper in hallways, rapairing bullatin boards), 

- Obtains nacessary materials when things need to be fixed for on^ 
going daily functions in tha building, 

- Sets up a graffiti board in the restrooms to minimize vandalism. 



Examples of Behavior of Individuals ^o are Usually Rated "Low" on 
^his Dimension i ' 

*- Unilatarally detarmines which school facilities are to be improved, 

^ Does not provide a safe storage place for expansive raatarial, 

* Never foresees equipment problems; reacts to peoples' requests and 
emergencies . 

^ Ignores a report of unsafe playground equipment antil something 
Dens to a child , 
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n 

RATING 



High 



* □ 



Average 



3 a 



2 a 



Low 



DO NOT 
WRITE IN 
THIS AREA 
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15, STRUCTURIS COhMUNICATlON WHICH PROVIDE FOR 
COOPERATION AMONG VARIOUS GROUPS IN SCHOOL 



CHECK ONLY ONE 
^RESPONSE 



If 70U hava InsufficienC informacion to rate this dimension^ "check 
this boxe 



" Sets up coramunieat ion devices so that teachers as a group can con-^ 
vey needs to support serviee personnel (e.g.j l-'^brarian, cafeteria 
staffs media specialises). 



Examples^ o ^Behavior of Individuals who are Usually Rated ''Average" 
on this Dimension ; 

^ Distributes a weekly newsletter to inform staff of school events* 

—Keeps appointment calendar on desk so anyone can make an appointment 
at any t ime . 

^ Trains students to make announcements of events each day* 

^ Meets with whole staff once a month with no agenda^ to discuss prob-- 
lems that have developed* 



Exmples of Behavior of Individuals who are Usually Rated "Low'' on 
this Dimension I 

^ Sees people only by appointment, 

^ Makes decisions without involving those whom it directly affects. 
^ Organises an Advisory Coimiiittee but never goes to a meeting. 

- Coimnunicates only by mail^ P, A* , or memo with staff and students. 

- Stays in office all day* 

20 8 7 



RATING 



Examples of Behavior of Individuals who are Usually Rated "High" on 
^his Dimensions 

^ Plans meetings of staff j supervisors j and parents to air concerns 
regarding school programs or problems. 

^ Communicates through meetings^ memos j or personal coimnunieat ions to 

tell people about school or things that impact on school's function^ 5 
ing, 

^ When dealing with irate or concerned parents j consults with the per- 
sonnel involved* 



High 



* □ 



Average 



Low 



DO NOT 
WRITE IN 
THIS AREA 
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SECTION Ci EFFECTIVENESS SCALE 



88 



ERIC 



EFFECTIVENISS SCALE 



Instrug t ionB t Various dimens iens &f sehoo 1 adminis trat or i are 1 is tad 
below. Raad eaeh oaa carefully and rata the inewbant by making a 
chaok mark at tha scala ^alua i^ieh rapresents your Jud^ant of \^ac 
tha person is really like. If you are not able to giva a reliable 
racing on a particular dimension baeausa you do not ha^e enough in^ 
foraation, mark tha "DO NOT KNOW*' box. 



1 . Adaptability ; 

How well does this person adapt his or her behavior and approach in 
dealing with different situations and different paopla? 



DO NOT 
WRITE IN 
THIS AREA 



do not know 



5 axtramaly wall 

4 above average 

3 satisfactory 

2 naada improvement 

1 falls short of requiramants 



2 . Confarring with Parents : 

How effactivaly does this person daal with parent confarances where 
he or sha naeds to review maCarial relating to a student experiencing 
a problem at school, to meet with tha parent (s) to discuss the stu^ 
dent's problem in an attempt to ramady the situations ^nd perhaps to 
prepare a brief written suramary of the outcome of the meeting for 
his or her immediate supervisor? 



do not know 



5 extremely well 

4 above average 

I 3 satisfactory 

I 2 needs improvement 

I 1 falls short of requirements 
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3 , Decision Making : 

How logical and sound are the judgments this person makes in uaing 
resources^ determining courses of action ^nd defining solutions to 
probleme? 



do not know 



I 5 extremely well 

j 4 abo\re av.^rage 

3 satisfactory 

j 2 needs improvement 

I 1 falls short of requirementi 



Dee is iveness : 



How firm and conclusive is this person in making decisions ^ rendering 
judgments J taking action and defanding decisions, judgments and ac- 
tions whan challenged? 



do not know 



5 extremely well 
4 above average 



I 3 satisfactory 

^ j 2 needs mprovement 

1 falls short of requiraments 



5 • Intarpersonal i 

To what extant is this person sensitive to others and behaves in a 
manner which reflects an appreciation of the needs , fealings and 
capabilities of others? How affectively does he or she show cour- 
tesy, tact and understanding , regardlags of a person's status or 
positionj accept interpersonal differences and develop rapport with 
others? 



do not know 



I 5 extremely well 

_ j 4 above average 

3 satisfactory 

I 2 needs improvament 

I 1 falls short of requirements 



DO NOT 
WRITE IN 
THIS AREA 
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6 • Managemeni Skill s 

How much leadarship does this person provide through direct ing and 
coordinating the activities of others ^ by delegating authority and 
responsibility and in providing the means to follovp-up? 

do not know 



5 extremely well 

I 4 above average 

I 3 aatiafactory 

2 needs improvement 
1 falls short of requirements 



7 . Observing Teaehera I 

How effeetively does this person use teacher obgervation skills by 
reviewing the teacher's background^ observing the teacher instructing 
a class, completing an Observation Form and subsequently meeting with 
the teacher to give feedback on the performance observed? 



do not know 



__ I 5 extremely well 
[ 4 above average 
[ 3 sat is iJactory 

I 2 needs improvement 

1 falls short of requirements 



S , Oral Coimnunicat ions ; 

How clearly and effectively does this person speak or express infor= 
mat ion orally , making proper use of techniques p such as voice inflec= 
tion^ eye contact ^ and good grammar and vocabulary? 

do not know 



DO NOT 
WRITE IN 
THIS AREA 
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5 extremely well 
_ I 4 above average 

3 satisfactory 

_^ I 2 needs improvement 

j 1 falls short of requirements 
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9 . Paper Wor k: 

How effectively does this person structure.^ organize and deal with 
memos, letters, correspondence and ocher reporte which vary in their 
importance and urgency; where he or she must write letters , memos , 
give instructions andj from time to time, clarify or explain action 
taken? 

do not know 



[ 5 extremely well 

I 4 above average 
3 satisfactory 
2 needs improvement 



I 1 falls short of requirements 



10, 



? 1 ann ing and Or ganig ing i 



How well does this person establish object tves, schedules , pribrities 
and strategies for hiraself or herself, as %^11 as others, to accom= 
plish specific results? 



do not know 



5 eKtremely well 

4 above average 

3 satisfactory 

2 needs improvement 

1 falls short of' requirements 



11, Perceptions 



How well does this person identify the critical elements of a situa^ 
tion, interpret implications of alternative courses of action and 
evaluate factors essential to the solution of a problem? 



do not know 



5 extremely well 

4 above average 

3 satisfactory 

2 needs improvement 

1 falls short of requirements 



DO NOT 
WRITE IN 
THIS AREA 
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1 2 . Leadership ? 

How effectively does this person lead and motivate students, parents, 
teachers, school officials and others? Does he or she receive loy= 
alty from them and inspire in them a sense of purpose and desire to 
do what they are asked? 

do not know 



5 extreraely well 

I 4 above average 

^ I 3 satisfaeCory 

I 2 needs improvement 
1 falls short of requirements 



13 1 Technical Know-how i 

How effective is this person in acquiring, utilising, and applying 
the technical knowledge required by his or her current position? 



do not know 



_ I 5 eKtremely well 
4 above average 
3 satisfactory 
2 needs improvement 
1 falls short of requirements 



14. Written Comnunicat ion ; 

How well does this person express inforrfiation effectively and clearly 
in writing, making proper use of grammar and vocabulary? 



do not know 



I I 5 eKtremely well 

I I 4 above average 

3 satisfactory 
2 needs improvement 
1 falls short of requirements 



DO NOT 
WRITE IN 
THIS AREA 
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15. 



In 



Overall Ratinj 



domparison with all others presently holding the same job, how 
would you rate this person in overall effectiveness in his or her 
position? 



DO NOT 
WRITE IN 
THIS AREA 



Rat ing 



^Canine 

9 

8 

7 
6 
5 
4 
3 
2 
1 



%ile 

96-100 

89-95 

80-88 

61-79 

40-60 

21-39 

12-20 

5-11 

1-4 



16, 



If 



Remarks i 



you wish to eonmient on the Effectiveness Seale or offer any sug- 
gastionSj please do so. 
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The information you have provided on this rating instrument will be 
used to evaluate the selection procedure of aehool level administra*- 
tore* Hopafullyi through our mutual efforts the selection process 
can be improved. Thank you for your cooperation* 



Please return the completed rating instrument tos 



Mr, Joe Goraeg 

Board Administration Bldg, 

Mail Code 9999 

Room 800 

Use the enclosed return envelope. 
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The SchDoi Board of Dade Florl^^s adheres to s palmy of 

nondiicrlminstlon In edue^tl^nif pfoprami/^te^tivltlts ind implQyment 
and itrives affirmitlvily to pro^ljitpl opp^^^unlty for all as required 
byi 

Titl© VI of thi Civil Righti ^etof 19S^^— prohlblfr ^vlserlminatisn 
on the basis of ra^f Oolor, fdim, or f^l^^iQnil origin. 

Title VII of the Civil Righ^teof 105^^, ai emended ^prohlblti 
disarlmlnation In empiovfri^ritsiithe b^i^Ss of raea^ eolor, religion^ 
sex^ or national orlsln^ 

Title IX of the EducatiQfi Amindm«tt^te of 1072 - prohibits 
dlsarlmlnatlon on the ^%\% pliiXi 

Age Diserlmination Aot 1B7# as i^^nended - prohibits dii- 
crimination on the batis of ggiMean 4^ C and 70, 

Motion 604 of the RihabilM AeC ^Df 1873 - prohibits dis- 
orimlnation agiinit tht hanjlsipped, 

Florida Educational EquHy^^t ^ pr^^^alblti discrimination on 
the bills of raoe^ seic, natiQrtlQHgin, r^^arltal status or handioap 
against a student or ertiploVgi. 

Veterani are provided re^mpi^yniint flgh^ ^Sn aaeordance with P,L, 
93-608 (Federal) and Seotion 0Uh Plpfi^da Statutei, whioh ilso 
siipulates categonoal prefertnoi^ f^fimployniQir^nt, 





